From 3ae9d0a884f948e8bd4e231982c287d5cfdab74e Mon Sep 17 00:00:00 2001 From: Scott Idem Date: Wed, 6 May 2026 10:39:42 -0400 Subject: [PATCH] Refine IDAA Jitsi reports UX Add Novi UUID exclusion and known-meeting filtering, default the report date range to the last 60 days, and hide Room Name unless global edit mode is enabled. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../CLIENT__IDAA_and_customized_mods.md | 52 ++++--- documentation/TODO__Agents.md | 5 +- .../ae_comp__site_config_editor.svelte | 42 ++++-- src/lib/ae_reports/reports_functions.ts | 107 ++++++++++++-- .../idaa/(idaa)/jitsi_reports/+page.svelte | 135 +++++++++++++----- 5 files changed, 253 insertions(+), 88 deletions(-) diff --git a/documentation/CLIENT__IDAA_and_customized_mods.md b/documentation/CLIENT__IDAA_and_customized_mods.md index 710a438c..1a9fda64 100644 --- a/documentation/CLIENT__IDAA_and_customized_mods.md +++ b/documentation/CLIENT__IDAA_and_customized_mods.md @@ -183,11 +183,16 @@ fetch(`${api_root_url}/customers/${uuid}`, { - **`novi_bb_base_url`**: (optional) Base URL used to build links for Bulletin Board notification emails. -- **`jitsi_exclude_names`**: (optional) Array of display name strings to exclude from Jitsi Reports. - Used to hide known staff and test accounts from the activity report so participant counts and lists - reflect real member activity only. Names are matched case-insensitively against `final_participants[].displayName`. - Example: `["Scott I.", "Michelle V.", "Brie K."]`. The filter is applied before the "Real meetings only" - threshold check — a session with only excluded participants is treated as having 0 real participants. +- **`jitsi_exclude_uuids`**: (optional) Array of Novi UUIDs to exclude from Jitsi Reports. + This is the canonical staff/test filter. UUIDs are matched case-insensitively against + `final_participants[].novi_uuid` when present. Example: `["uuid-1", "uuid-2"]`. + +- **`jitsi_known_meetings`**: (optional) Array of meeting names / room names to keep in the report. + When this list is non-empty, only matching `room_name` values are shown. Matching is + case-insensitive. + +- **Legacy fallback:** `jitsi_exclude_names` is still honored for older configs, but it should be + migrated to UUIDs. - **Email config values** (`noreply_email`, `noreply_name`, `admin_email`, `admin_name`): used by functions that send notification emails (BB posts, comments, recovery meetings). @@ -461,7 +466,7 @@ Moderation permissions are controlled by `novi_jitsi_mod_li` in the IDAA store. An admin/staff reporting tool that aggregates raw Jitsi activity logs into human-readable meeting sessions. It is **not** a member-facing page — IDAA members do not see it. -**Reminder:** this page is still incomplete. We still need the Novi UUID filter to work and we still need meeting-name whitelist filtering. +**Reminder:** this page now filters staff by Novi UUID and can whitelist known meeting names from site config. ### View Modes @@ -480,30 +485,33 @@ Both modes use the same filtered data set — switching views does not reset fil | --- | --- | --- | | **Real meetings only** | off | Show only sessions where `real_participant_count >= 2` OR `duration > 5 min`. Applied **after** staff exclusion (see below). | | **Min. Participants** | 0 | Minimum `real_participant_count` to display a session. | -| **Room Name** | (empty) | Case-insensitive substring match against `room_name`. | -| **From / To** | (empty) | Date range applied to `start_time`. "To" date includes the full end of day. | +| **Room Name** | edit mode only | Case-insensitive substring match against `room_name`. Hidden unless AE global edit mode is on. | +| **From / To** | last 60 days / today | Date range applied to `start_time`. "To" date includes the full end of day. | A "Reset Filters" button appears whenever any filter is non-default. -### Staff / Test Exclusion +### Staff / Meeting Filtering -**Problem:** Staff and test accounts (Scott, Michelle, Brie) join real member meetings for setup, testing, and tech support. Their presence inflates participant counts and pollutes the participant list. +**Problem:** Staff/test accounts and one-off test rooms distort the reports. -**Solution:** A configurable exclusion list in `site_cfg_json`: +**Site config keys:** ```json -{ "jitsi_exclude_names": ["Scott I.", "Michelle V.", "Brie K."] } +{ + "jitsi_exclude_uuids": ["uuid-1", "uuid-2"], + "jitsi_known_meetings": ["IDAA-BIPOC-Meeting", "IDAA-Sunday-Meeting"] +} ``` -**How it works (client-side only, no backend change needed):** +**How it works:** -1. On load, the page reads `$ae_loc.site_cfg_json?.jitsi_exclude_names` (string array, defaults to `[]`). -2. For every session, a `real_participants` derived list is computed by filtering `final_participants` against the exclusion list (case-insensitive display name match). -3. `real_participant_count = real_participants.length` — this count drives all filters, stats, and the participant list column in grouped view. -4. The raw `final_participant_count` from the API is never shown to the user once an exclusion list is configured. +1. The page reads `$ae_loc.site_cfg_json?.jitsi_exclude_uuids` and excludes matching participants by Novi UUID. +2. If a participant record does not include a UUID in the activity log, it is left visible; UUIDs are used whenever available. +3. `real_participant_count = real_participants.length` drives filters, exports, and the per-meeting attendee count. +4. Room-level unique participant counts are computed from Novi UUIDs when present, with display-name fallback only for UUID-less records. +5. If `$ae_loc.site_cfg_json?.jitsi_known_meetings` is non-empty, only meetings whose `room_name` matches one of the listed names are shown. +6. The Room Name filter is only shown when global edit mode is enabled. -**Why display-name matching (not Novi UUID):** Jitsi participant data (`meta_json.participants`) only contains `displayName` and `role` — the Novi UUID is not passed through to the activity log. UUID-based exclusion would require a Jitsi config change plus a backend schema update and is deferred. Display names for OSIT staff are stable and controlled. - -**Still pending:** the new Jitsi Reports page still needs a real Novi UUID filter and a whitelist of meeting names so staff can narrow the report set without relying only on display names. +**Note:** matching is case-insensitive on the stored `room_name` / meeting name. ### Summary Stats @@ -514,7 +522,7 @@ Shown above the meeting list when data is loaded. Stats reflect the **filtered + - **Avg Duration** — mean session duration (HH:MM:SS) - **Total Duration** — sum of all session durations (HH:MM:SS) -In grouped view, each room header also shows its own subtotals (meeting count, unique participants). +In grouped view, each room header also shows its own subtotals (meeting count, unique participants by Novi UUID when available). ### Jitsi URL Builder @@ -733,4 +741,4 @@ ae_loc.idaa_loc = { novi_uuid: 'test-uuid-value', ... }; --- **Document Status:** ✅ Current -**Last Verified:** 2026-05-05 — added Module 5: Jitsi Reports (grouped view, real-meetings filter, staff exclusion via `jitsi_exclude_names`); fixed route tree (`jitsi_reports/` is inside `(idaa)/`) +**Last Verified:** 2026-05-06 — added Module 5: Jitsi Reports (grouped view, UUID exclusion, known-meeting whitelist, UUID-based unique counts); fixed route tree (`jitsi_reports/` is inside `(idaa)/`) diff --git a/documentation/TODO__Agents.md b/documentation/TODO__Agents.md index 7edadc6c..bf877151 100644 --- a/documentation/TODO__Agents.md +++ b/documentation/TODO__Agents.md @@ -185,9 +185,8 @@ suddenly jumps to 0 errors, verify it's not because a bad `.d.ts` replaced a pac effect without re-login. ### [IDAA] Jitsi Reports still incomplete -- [ ] **Finish Jitsi Reports filters** — the new Jitsi Reports page still needs a working Novi UUID - filter and meeting-name whitelist filtering so staff can narrow the report set without relying on - display-name matching alone. +- [x] **Finish Jitsi Reports filters** — added Novi UUID exclusion plus meeting-name whitelist + filtering, with room-level unique counts based on Novi UUID when present. (2026-05-06) ### [PWA] Service worker ignoring `chrome-extension://` requests Browser console shows repeated errors: diff --git a/src/lib/ae_core/ae_comp__site_config_editor.svelte b/src/lib/ae_core/ae_comp__site_config_editor.svelte index 620c34ec..83fa999e 100644 --- a/src/lib/ae_core/ae_comp__site_config_editor.svelte +++ b/src/lib/ae_core/ae_comp__site_config_editor.svelte @@ -51,9 +51,9 @@ let show_llm_api_token = $state(false); // Ensure we have a valid object if (!cfg_json) cfg_json = {}; -function add_to_list(key: string) { +function add_to_list(key: string, prompt_label: string) { if (!cfg_json[key]) cfg_json[key] = []; - const val = prompt('Enter Novi UUID:'); + const val = prompt(prompt_label); if (val) cfg_json[key].push(val); } @@ -61,6 +61,10 @@ function remove_from_list(key: string, index: number) { cfg_json[key].splice(index, 1); } +function list_count(key: string): number { + return Array.isArray(cfg_json[key]) ? cfg_json[key].length : 0; +} + // Sync Raw JSON string when entering the tab $effect(() => { if (active_tab === 'raw') { @@ -365,24 +369,32 @@ $effect(() => {
- {#each [{ key: 'novi_admin_li', label: 'Novi Admins', color: 'text-error-500' }, { key: 'novi_trusted_li', label: 'Novi Trusted', color: 'text-warning-500' }, { key: 'novi_jitsi_mod_li', label: 'Jitsi Moderators', color: 'text-primary-500' }, { key: 'novi_idaa_group_guid_li', label: 'Member Group GUIDs', color: 'text-secondary-500' }] as list (list.key)} -
-
- {list.label} + {#each [{ key: 'novi_admin_li', label: 'Novi Admins', color: 'text-error-500', prompt: 'Enter Novi UUID:' }, { key: 'novi_trusted_li', label: 'Novi Trusted', color: 'text-warning-500', prompt: 'Enter Novi UUID:' }, { key: 'novi_jitsi_mod_li', label: 'Jitsi Moderators', color: 'text-primary-500', prompt: 'Enter Novi UUID:' }, { key: 'novi_idaa_group_guid_li', label: 'Member Group GUIDs', color: 'text-secondary-500', prompt: 'Enter Group GUID:' }, { key: 'jitsi_exclude_uuids', label: 'Jitsi Excluded UUIDs', color: 'text-error-500', prompt: 'Enter Novi UUID to exclude:' }, { key: 'jitsi_known_meetings', label: 'Known IDAA Meetings', color: 'text-primary-500', prompt: 'Enter meeting name to allow:' }] as list (list.key)} +
+ + + {list.label} + ({list_count(list.key)}) + -
-
- {#each cfg_json[list.key] ?? [] as uuid, i (uuid)} + +
+ {#each cfg_json[list.key] ?? [] as item, i (i)}
- {uuid} + {item}
{/each}
-
+ {/each}
diff --git a/src/lib/ae_reports/reports_functions.ts b/src/lib/ae_reports/reports_functions.ts index 24086bc2..f5ec3f82 100644 --- a/src/lib/ae_reports/reports_functions.ts +++ b/src/lib/ae_reports/reports_functions.ts @@ -24,6 +24,43 @@ function parse_duration_seconds(d: string): number { return (h || 0) * 3600 + (m || 0) * 60 + (s || 0); } +function normalize_text(value: unknown): string { + return typeof value === 'string' ? value.trim() : ''; +} + +function normalize_uuid(value: unknown): string { + return normalize_text(value).toLowerCase(); +} + +function extract_participant_uuid(source: Record): string { + const nested_user = source.user as Record | undefined; + const nested_context = source.context as Record | undefined; + const nested_context_user = nested_context?.user as + | Record + | undefined; + + const candidates = [ + source.novi_uuid, + source.uuid, + source.novi_customer_uid, + nested_user?.novi_uuid, + nested_user?.uuid, + nested_user?.id, + nested_context_user?.novi_uuid, + nested_context_user?.uuid, + nested_context_user?.id, + source.user_id, + source.id + ]; + + for (const candidate of candidates) { + const uuid = normalize_uuid(candidate); + if (uuid) return uuid; + } + + return ''; +} + /** * @description Queries all Jitsi-related activity logs and processes them into a structured report, * grouped by meeting ID. @@ -118,14 +155,16 @@ export async function qry__jitsi_report({ // 1. jitsi_meeting_participant_joined events: meta.full_name (all who ever joined) // 2. jitsi_meeting_init / stats / end snapshots: meta.participants[].displayName+role // Source 2 has role info; source 1 may catch participants who left before the snapshot. - // We merge both into a Map per meeting, deduplicating by name. + // We merge both into a Map per meeting, keeping the + // stable display-name key used by the existing report while preserving Novi UUIDs + // when the log payload includes them. // // Duration: take the MAX across all init/stats/end events — periodic stats may // have a higher value than the final init summary in some Jitsi configurations. const meetings = new Map(); - // Per-meeting participant map: displayName → role - const participant_maps = new Map>(); + // Per-meeting participant map: displayName → participant record + const participant_maps = new Map>(); // Per-meeting max observed duration in seconds const max_duration_secs = new Map(); @@ -177,15 +216,35 @@ export async function qry__jitsi_report({ // Merge snapshot participant list (has role info — preferred source) const snapshot = meta.participants as - | Array<{ displayName?: string; role?: string }> + | Array> | undefined; if (Array.isArray(snapshot)) { for (const p of snapshot) { - if (!p.displayName) continue; - // Only overwrite an existing entry if we're upgrading from participant → moderator - const existing_role = p_map.get(p.displayName); - if (!existing_role || (existing_role !== 'moderator' && p.role === 'moderator')) { - p_map.set(p.displayName, p.role ?? 'participant'); + const display_name = normalize_text(p.displayName); + if (!display_name) continue; + + const role = + typeof p.role === 'string' ? p.role : 'participant'; + const novi_uuid = extract_participant_uuid(p); + const existing_participant = p_map.get(display_name); + + if (!existing_participant) { + p_map.set(display_name, { + displayName: display_name, + role, + ...(novi_uuid ? { novi_uuid } : {}) + }); + continue; + } + + if ( + existing_participant.role !== 'moderator' && + role === 'moderator' + ) { + existing_participant.role = 'moderator'; + } + if (!existing_participant.novi_uuid && novi_uuid) { + existing_participant.novi_uuid = novi_uuid; } } } @@ -193,9 +252,30 @@ export async function qry__jitsi_report({ // Collect participants from join events (may catch people who left before the snapshot) if (log.action === 'jitsi_meeting_participant_joined') { - const full_name = (meta.full_name as string) || ''; - if (full_name && !p_map.has(full_name)) { - p_map.set(full_name, (meta.role as string) ?? 'participant'); + const display_name = normalize_text(meta.full_name); + if (display_name) { + const role = + typeof meta.role === 'string' ? meta.role : 'participant'; + const novi_uuid = extract_participant_uuid(meta); + const existing_participant = p_map.get(display_name); + + if (!existing_participant) { + p_map.set(display_name, { + displayName: display_name, + role, + ...(novi_uuid ? { novi_uuid } : {}) + }); + } else { + if ( + existing_participant.role !== 'moderator' && + role === 'moderator' + ) { + existing_participant.role = 'moderator'; + } + if (!existing_participant.novi_uuid && novi_uuid) { + existing_participant.novi_uuid = novi_uuid; + } + } } } @@ -217,8 +297,7 @@ export async function qry__jitsi_report({ if (!meeting_report) continue; if (p_map.size > 0) { // Sort: moderators first, then alphabetically - meeting_report.final_participants = Array.from(p_map.entries()) - .map(([displayName, role]) => ({ displayName, role })) + meeting_report.final_participants = Array.from(p_map.values()) .sort((a: any, b: any) => { if (a.role === 'moderator' && b.role !== 'moderator') return -1; if (a.role !== 'moderator' && b.role === 'moderator') return 1; diff --git a/src/routes/idaa/(idaa)/jitsi_reports/+page.svelte b/src/routes/idaa/(idaa)/jitsi_reports/+page.svelte index 5f78fc2f..34370f02 100644 --- a/src/routes/idaa/(idaa)/jitsi_reports/+page.svelte +++ b/src/routes/idaa/(idaa)/jitsi_reports/+page.svelte @@ -15,6 +15,7 @@ interface MeetingEvent { interface MeetingParticipant { displayName: string; role: string; + novi_uuid?: string; } interface MeetingReport { @@ -39,6 +40,45 @@ let meetings_loading = $state(true); let meetings_error = $state(null); let meetings_load_started = $state(false); +function normalize_text(value: unknown): string { + return typeof value === 'string' ? value.trim() : ''; +} + +function normalize_uuid(value: unknown): string { + return normalize_text(value).toLowerCase(); +} + +function normalize_meeting_name(value: unknown): string { + return normalize_text(value).toLowerCase(); +} + +function normalize_list(value: unknown): string[] { + return Array.isArray(value) + ? value + .map((item) => normalize_text(item)) + .filter((item) => item.length > 0) + : []; +} + +function participant_identity(p: MeetingParticipant): string { + const uuid = normalize_uuid(p.novi_uuid); + if (uuid) return `uuid:${uuid}`; + return `name:${normalize_meeting_name(p.displayName)}`; +} + +function format_date_input_value(date: Date): string { + const year = date.getFullYear(); + const month = `${date.getMonth() + 1}`.padStart(2, '0'); + const day = `${date.getDate()}`.padStart(2, '0'); + return `${year}-${month}-${day}`; +} + +const today = new Date(); +const default_filter_date_to = format_date_input_value(today); +const default_filter_date_from = format_date_input_value( + new Date(today.getFullYear(), today.getMonth(), today.getDate() - 60) +); + $effect(() => { if (meetings_load_started) return; if (!$idaa_loc.novi_verified && !$ae_loc.trusted_access) return; @@ -70,13 +110,29 @@ $effect(() => { }); // --- Staff/test exclusion --- -// Read display names to exclude from site_cfg_json.jitsi_exclude_names. -// Matched case-insensitively against MeetingParticipant.displayName. -let exclude_names = $derived( - (($ae_loc.site_cfg_json as Record) - ?.jitsi_exclude_names as string[] | undefined) - ?.map((n) => n.toLowerCase().trim()) ?? [] +// UUID-based exclusion is the canonical path; the legacy name list stays as a fallback +// for older site config values until those are cleaned up. +let exclude_uuids = $derived( + normalize_list( + ($ae_loc.site_cfg_json as Record) + ?.jitsi_exclude_uuids + ).map((uuid) => uuid.toLowerCase()) ); +let exclude_names = $derived( + normalize_list( + ($ae_loc.site_cfg_json as Record) + ?.jitsi_exclude_names + ).map((name) => name.toLowerCase()) +); +let known_meetings = $derived( + normalize_list( + ($ae_loc.site_cfg_json as Record) + ?.jitsi_known_meetings + ).map((name) => name.toLowerCase()) +); +let exclude_uuid_set = $derived(new Set(exclude_uuids)); +let exclude_name_set = $derived(new Set(exclude_names)); +let known_meeting_set = $derived(new Set(known_meetings)); // Apply exclusion to every meeting — produces the "real" participant list // used by all downstream filters, stats, and display. @@ -84,15 +140,19 @@ let meetings_enriched = $derived( meetings_all.map((m) => { // final_participants can be null/undefined if meta_json.participants was null in the API response const all_participants: MeetingParticipant[] = m.final_participants ?? []; - const real_participants = - exclude_names.length > 0 - ? all_participants.filter( - (p) => - !exclude_names.includes( - p.displayName.toLowerCase().trim() - ) - ) - : all_participants; + const real_participants = all_participants.filter((p) => { + const participant_uuid = normalize_uuid(p.novi_uuid); + if (participant_uuid && exclude_uuid_set.has(participant_uuid)) { + return false; + } + + const participant_name = normalize_meeting_name(p.displayName); + if (exclude_name_set.has(participant_name)) { + return false; + } + + return true; + }); return { ...m, real_participants, @@ -108,23 +168,23 @@ let group_by_room = $state(true); let filter_real_only = $state(false); let filter_min_participants = $state(0); let filter_room_name = $state(''); -let filter_date_from = $state(''); -let filter_date_to = $state(''); +let filter_date_from = $state(default_filter_date_from); +let filter_date_to = $state(default_filter_date_to); let filters_are_modified = $derived( filter_real_only || filter_min_participants !== 0 || filter_room_name !== '' || - filter_date_from !== '' || - filter_date_to !== '' + filter_date_from !== default_filter_date_from || + filter_date_to !== default_filter_date_to ); function reset_filters() { filter_real_only = false; filter_min_participants = 0; filter_room_name = ''; - filter_date_from = ''; - filter_date_to = ''; + filter_date_from = default_filter_date_from; + filter_date_to = default_filter_date_to; } // --- Duration helpers --- @@ -154,6 +214,11 @@ function compute_end_time(start_time: string, duration: string): string { // All counts and thresholds use real_participant_count (post-exclusion). let meetings_filtered = $derived.by(() => { return meetings_enriched.filter((m) => { + if (known_meeting_set.size > 0) { + if (!known_meeting_set.has(normalize_meeting_name(m.room_name))) { + return false; + } + } if (filter_real_only) { const dur_secs = parse_duration_seconds(m.final_duration); if (m.real_participant_count < 2 && dur_secs <= 300) return false; @@ -230,7 +295,7 @@ function room_stats(sessions: MeetingReportEnriched[]) { 0 ); const unique_names = new Set( - sessions.flatMap((m) => m.real_participants.map((p) => p.displayName)) + sessions.flatMap((m) => m.real_participants.map((p) => participant_identity(p))) ); return { meeting_count: sessions.length, @@ -424,17 +489,19 @@ function export_json() { class="border-surface-200-800 bg-surface-50-950 w-20 rounded border px-2 py-1" />
- - + {#if $ae_loc.edit_mode} + + + {/if}