diff --git a/README.md b/README.md index 5cbce62f..f6851202 100644 --- a/README.md +++ b/README.md @@ -163,6 +163,33 @@ npm run deploy:remote:prod - Private runtime variables are passed via the Docker Compose `.env` file in `aether_container_env/`. - **Remote deploy**: `aether_container_env/deploy.sh` handles git pull + Docker build + restart on the server. Triggered via `npm run deploy:remote:*`. +### Client-Side Cache & IDB Version Management + +The app uses Dexie (IndexedDB) as a local cache for API data (SWR pattern). To prevent +stale cached records from persisting across deploys, two version-tracking systems exist +in `src/lib/stores/store_versions.ts`: + +**localStorage store versions (`AE_LOC_VERSION`, etc.)** +Track the schema of persisted Svelte stores (`ae_loc`, `ae_events_loc`, etc.). +Bump when a store's shape changes in a breaking way (field type change, required rename). +The check runs synchronously at module import time, before any store hydrates. + +**IDB content versions (`IDB_CONTENT_VERSIONS`)** +Track the content shape of Dexie table rows — specifically what `properties_to_save` +writes to each table. Bump when `properties_to_save` in an object file changes in a way +that makes existing cached rows stale (fields added/removed/renamed, computed field behavior +changed). The `check_and_clear_idb_table()` helper reads a localStorage key per table and +clears the Dexie table on mismatch. Call it from the module's layout on mount. + +**When to bump `IDB_CONTENT_VERSIONS`:** +If you change `properties_to_save` in `ae_events__event.ts` (or any other object file), +bump the matching entry here. Failure to do so has historically caused silent "no data" +states that are extremely difficult to diagnose — stale rows pass silently, filter to zero, +and the error looks identical to a genuinely empty result. + +Currently wired: `events.event` (via `src/routes/idaa/(idaa)/+layout.svelte`). +All other tables are defined but not yet wired — see the comment block in `store_versions.ts`. + --- ## Developing (Local HMR) diff --git a/documentation/BOOTSTRAP__AI_Agent_Quickstart.md b/documentation/BOOTSTRAP__AI_Agent_Quickstart.md index e0536527..cb150c71 100644 --- a/documentation/BOOTSTRAP__AI_Agent_Quickstart.md +++ b/documentation/BOOTSTRAP__AI_Agent_Quickstart.md @@ -336,6 +336,38 @@ These are real incidents — know them before you start. replacing the local IDB result set. For empty text searches, the full local result set should drive the display; server refreshes should update cache, not shrink visibility. +13. **Not bumping `IDB_CONTENT_VERSIONS` when changing `properties_to_save`** — this caused + the IDAA Recovery Meetings "no meetings found" bug for approximately one year (2025–2026). + + **What happened:** A deploy changed `properties_to_save` in `ae_events__event.ts`, but no + one bumped `IDB_CONTENT_VERSIONS.events.event` in `store_versions.ts`. Existing users kept + the old stale event records in IndexedDB indefinitely. On the Recovery Meetings page, the + fast path (IDB search) returned those stale records, which all failed the `account_id` + filter and returned 0 results. The API call then either errored silently or was filtered + to 0 by the secondary client-side filter. Critically, the error state and the genuinely + empty state showed the **same** "No meetings found" message — users and staff had no + indication a failure had occurred. The manual Full Reset (via the `?` help panel) always + fixed it, but no one knew why it worked, making the root cause impossible to track down. + + **The fix (2026-05-16):** `check_and_clear_idb_table()` in `store_versions.ts` is now + wired in `src/routes/idaa/(idaa)/+layout.svelte` for `db_events.event`. On a version + match it costs one localStorage read. On a mismatch it silently clears the table; the + SWR pattern then repopulates from the API on next load. + + **The rule going forward:** + - When you change `properties_to_save` in any `ae_events__*.ts` file (or any other + object file) in a way that makes existing cached records stale — fields added, removed, + renamed, or where a computed field's behavior changes — **bump the matching entry in + `IDB_CONTENT_VERSIONS` in `src/lib/stores/store_versions.ts`**. + - If the table is not yet wired, wire it first (see the wiring instructions in the + `IDB_CONTENT_VERSIONS` comment block in `store_versions.ts`). + - Currently wired: `events.event`. All other tables are not yet wired. + + **Also:** Never show the same UI message for both a failed API call and a genuinely empty + result. Always distinguish `qry__status === 'error'` from `qry__status === 'done'` with + 0 results in your templates. Silent failures look like data problems and are extremely + difficult to diagnose. + --- ## 8. Source Layout (Quick Reference) diff --git a/src/lib/stores/store_versions.ts b/src/lib/stores/store_versions.ts index c4aaefa1..2a4194cc 100644 --- a/src/lib/stores/store_versions.ts +++ b/src/lib/stores/store_versions.ts @@ -38,29 +38,47 @@ export const AE_BADGES_LOC_VERSION = 1; // Added 2026-04-02: promoted from e export const AE_LEADS_LOC_VERSION = 1; // Added 2026-04-03: promoted from events_loc.leads /** - * IDB_CONTENT_VERSIONS — per-table Dexie content version tracking. + * IDB_CONTENT_VERSIONS — per-table Dexie (IndexedDB) content version tracking. * - * NOT YET ACTIVE. Wiring task tracked in TODO__Agents.md (post June 10). + * BACKGROUND: + * Stale IDB records persisting across deploys were the root cause of the "no meetings + * found" bug on the IDAA Recovery Meetings page — a ~1-year unresolved issue (2025-2026). + * After a deploy that changed properties_to_save, old cached event records stayed in IDB + * with wrong or missing fields. The fast path returned them, filtered to 0 (account_id + * mismatch), the API call errored silently, and the error state showed the same message + * as a genuinely empty result. Fixed 2026-05-16. * - * HOW IT WILL WORK (when wired): - * Each db_*.ts will call a helper (core__idb_dexie.ts) on open that checks a - * lightweight `_meta` table. If the stored version for a table doesn't match - * the constant here, the table is cleared and the version record is updated. - * The SWR pattern then repopulates from the API on next access. + * WIRING STATUS (as of 2026-05-16): + * events.event → ACTIVE — wired in src/routes/idaa/(idaa)/+layout.svelte + * All other tables → NOT YET WIRED (see instructions below) * - * HOW TO USE (once active): - * Bump a table's version here when properties_to_save changes in a way that - * makes existing cached records stale (e.g. adding/removing stored fields, - * changing computed field behavior). Do NOT bump for schema-only changes - * (new indexes, new tables) — those belong in db.version() Dexie migrations. + * HOW IT WORKS (for wired tables): + * check_and_clear_idb_table() (bottom of this file) is called on module/page load. + * It reads a localStorage key `ae_idb_ver____` and compares it + * to the version here. On mismatch, the Dexie table is cleared and the key is updated. + * The SWR pattern then repopulates from the API on next access. On a version match + * (the normal case after first load) the cost is a single localStorage read — free. + * + * HOW TO WIRE A NEW TABLE: + * 1. Import check_and_clear_idb_table from '$lib/stores/store_versions' in the relevant + * layout or page (the one that owns that module's data lifecycle). + * 2. Call it on mount, non-blocking: + * check_and_clear_idb_table(db_X.table_name, 'db_key', 'table_name').catch(() => {}); + * Use top-level `if (browser) { ... }` or an `$effect` with no reactive deps. + * 3. That's it. Bumping the version constant below is all that's needed for future clears. + * + * WHEN TO BUMP A VERSION: + * Bump when properties_to_save in an object file changes in a way that makes existing + * cached records stale — e.g. adding/removing stored fields, changing how a computed + * field (like tmp_sort_1) is built, or a backend change that alters field semantics. + * Do NOT bump for Dexie schema-only changes (new indexes, new tables) — those belong + * in db.version() Dexie migrations in the db_*.ts file. * * IDAA NOTES: - * IDAA tables (posts, archives) are already cleared aggressively via - * indexedDB.deleteDatabase() on sign-out and on auth failure in (idaa)/+layout.svelte. - * The content version check is a complementary mechanism for deploy-time resets, - * NOT a replacement for the auth-driven wipe. When wiring, ensure IDAA tables are - * only cleared on IDB open (not mid-session), and that the _meta table itself is - * also cleared when deleteDatabase() is called. + * IDAA tables (posts, archives, events) are already cleared aggressively via + * indexedDB.deleteDatabase() on auth failure in (idaa)/+layout.svelte. + * The content version check is a deploy-time complement — it handles the case where + * auth succeeds but IDB data is stale from a prior deploy. Both mechanisms are needed. */ export const IDB_CONTENT_VERSIONS = { journals: { @@ -68,7 +86,7 @@ export const IDB_CONTENT_VERSIONS = { journal_entry: 3 // 2026-05-14: removed content_md_html + history_md_html from properties_to_save }, events: { - event: 1, + event: 2, // Bumped 2026-05-16: force-clear stale IDB data causing "no meetings found" on IDAA event_session: 1, event_presenter: 1, event_badge: 1, @@ -135,3 +153,49 @@ function _check_and_wipe(key: string, expected_version: number): void { ); } } + +/** + * Checks the IDB table content version against the expected version in IDB_CONTENT_VERSIONS. + * If the stored version is stale (or missing), clears the entire table and records the new + * version so the next call is a no-op. + * + * @param db_table - A Dexie table reference (e.g. db_events.event) + * @param db_name - The top-level key in IDB_CONTENT_VERSIONS (e.g. 'events') + * @param table_name - The table key inside that group (e.g. 'event') + * + * WHY localStorage for version tracking (not a _meta IDB table): + * Simpler — no Dexie schema migration needed. localStorage and IDB are cleared together + * by Full Reset (e_app_help_tech.svelte), so they stay in sync across manual wipes. + * + * USAGE: call on mount, non-blocking. Example from (idaa)/+layout.svelte: + * check_and_clear_idb_table(db_events.event, 'events', 'event').catch(() => {}); + * + * After clearing, the SWR pattern in the relevant page automatically re-fetches from + * the API and repopulates the table. No explicit reload is needed. + */ +export async function check_and_clear_idb_table( + db_table: { clear(): Promise }, + db_name: string, + table_name: string +): Promise { + if (typeof localStorage === 'undefined') return; + + const versions = IDB_CONTENT_VERSIONS as Record>; + const expected_ver = versions[db_name]?.[table_name]; + if (expected_ver == null) return; + + const key = `ae_idb_ver__${db_name}__${table_name}`; + const stored_ver = parseInt(localStorage.getItem(key) ?? '0', 10); + + if (stored_ver !== expected_ver) { + try { + await db_table.clear(); + localStorage.setItem(key, String(expected_ver)); + console.info( + `[idb_versions] '${db_name}.${table_name}' cleared — v${stored_ver} → v${expected_ver}` + ); + } catch (e) { + console.warn(`[idb_versions] Failed to clear '${db_name}.${table_name}':`, e); + } + } +} diff --git a/src/routes/idaa/(idaa)/+layout.svelte b/src/routes/idaa/(idaa)/+layout.svelte index 667e942d..78a7b161 100644 --- a/src/routes/idaa/(idaa)/+layout.svelte +++ b/src/routes/idaa/(idaa)/+layout.svelte @@ -21,6 +21,7 @@ import { idaa_loc, idaa_sess, idaa_slct } from '$lib/stores/ae_idaa_stores'; import { db_posts } from '$lib/ae_posts/db_posts'; import { db_archives } from '$lib/ae_archives/db_archives'; import { db_events } from '$lib/ae_events/db_events'; +import { check_and_clear_idb_table } from '$lib/stores/store_versions'; interface Props { /** @type {import('./$types').LayoutData} */ @@ -63,6 +64,22 @@ let verify_in_flight = false; // Storing the failed UUID means only that exact UUID is skipped; any other UUID is a clean slate. let verify_failed_for_uuid: string | null = null; +// Clear stale db_events.event IDB data on IDAA session start. +// +// WHY: Stale cached event records were the root cause of the "no meetings found" bug +// on the IDAA Recovery Meetings page — a ~1-year unresolved issue (fixed 2026-05-16). +// After a deploy that changed properties_to_save, old IDB records persisted with missing +// or wrong fields. The search fast path returned 0 results (failed account_id filter), +// the API call errored silently, and the error state showed the same message as a real +// empty result — users had no indication anything was wrong. +// +// This runs once per IDAA session. On a version match (normal case) it costs one +// localStorage read. On a mismatch it clears the table; the SWR search re-fetches. +// To force a clear after a deploy: bump IDB_CONTENT_VERSIONS.events.event in store_versions.ts. +if (browser) { + check_and_clear_idb_table(db_events.event, 'events', 'event').catch(() => {}); +} + // Show a manual reset button if the spinner is still visible after this many ms. // Handles the case where site_cfg_json loads without novi_idaa_api_key (stale cache) // or the Novi API call hangs — the user would otherwise be stuck with no escape. diff --git a/src/routes/idaa/(idaa)/recovery_meetings/+page.svelte b/src/routes/idaa/(idaa)/recovery_meetings/+page.svelte index e14f0aa0..28c997d7 100644 --- a/src/routes/idaa/(idaa)/recovery_meetings/+page.svelte +++ b/src/routes/idaa/(idaa)/recovery_meetings/+page.svelte @@ -40,6 +40,78 @@ let event_id_li: Array = $state([]); let search_debounce_timer: any = null; let last_search_id = 0; let last_executed_key = ''; +// Auto-retry state: one silent background retry is attempted before showing the error +// state to the user. Manual "Try Again" button resets this so users can always retry. +// WHY: Mobile connections drop briefly. A single retry resolves most transient failures +// without the user ever seeing an error. If both attempts fail, surface the error clearly +// with a visible retry button — NOT the same "No meetings found" message a real empty +// result would show. (Conflating the two was a key reason the bug went unsolved so long.) +let auto_retry_count = 0; + +// ── Escape-hatch: cache-reset button after suspicious zero-result delay ────────────── +// +// WHY: The auto-retry + error state covers API failures (caught exceptions). But there +// is a separate failure mode where the API succeeds and returns 0 results — due to +// stale IDB data, a missed IDB_CONTENT_VERSIONS bump, or an undetected filtering issue. +// In that path qry__status lands on 'done', not 'error', so no retry or error UI fires. +// With ~140 active meetings in the database, zero results with no active filters is +// ALWAYS a symptom of something wrong. After 8 seconds in that state we surface a +// visible escape hatch. Using no-filters as the trigger prevents false positives when +// a member genuinely filters to a category that has no results (e.g. "Type = Workshop"). +// +// The reset clears db_events.event directly (bypasses the version check, which would be +// a no-op here since the version already matches after the initial layout-mount clear). +// The SWR search then re-fetches from the API and repopulates the table. + +// True only when: search is done, result list is empty, AND no filter dimensions are set. +let no_results_no_filters = $derived( + $idaa_sess.recovery_meetings.qry__status === 'done' && + event_id_li.length === 0 && + !$idaa_loc.recovery_meetings.qry__physical && + !$idaa_loc.recovery_meetings.qry__virtual && + !$idaa_loc.recovery_meetings.qry__type && + !($idaa_loc.recovery_meetings.qry__fulltext_str?.trim()) +); + +let show_cache_reset_btn = $state(false); +let cache_reset_timer: any = null; + +// Starts/cancels the 8-second countdown based on no_results_no_filters. +// $effect only re-runs when the derived VALUE changes (true↔false), not on every +// store write — so the timer counts down cleanly without spurious resets. +$effect(() => { + if (cache_reset_timer) { + clearTimeout(cache_reset_timer); + cache_reset_timer = null; + } + + if (no_results_no_filters) { + show_cache_reset_btn = false; + cache_reset_timer = setTimeout(() => { + show_cache_reset_btn = true; + }, 8000); + } else { + show_cache_reset_btn = false; + } + + return () => { + if (cache_reset_timer) clearTimeout(cache_reset_timer); + }; +}); + +async function handle_cache_reset() { + show_cache_reset_btn = false; + if (cache_reset_timer) { clearTimeout(cache_reset_timer); cache_reset_timer = null; } + auto_retry_count = 0; + try { + await db_events.event.clear(); + console.log('[recovery_meetings] Escape-hatch reset: db_events.event cleared'); + } catch (e) { + console.warn('[recovery_meetings] Escape-hatch reset: failed to clear IDB', e); + } + $idaa_sess.recovery_meetings.search_version++; +} +// ───────────────────────────────────────────────────────────────────────────────────── // Standardized Reactive Search Pattern (Aether UI V3) // This effect manages the orchestration between UI state and data fetching. @@ -294,12 +366,26 @@ async function handle_search_refresh(qry_key: string) { } catch (error) { if (current_search_id === last_search_id) { console.error('Revalidation failed:', error); - $idaa_sess.recovery_meetings.qry__status = 'error'; - // If the API failed (e.g. 400/500), we should NOT preserve results from a - // previous successful search as it's misleading. - // We clear the list so the 'No recovery meetings found' / Error message shows. - event_id_li = []; + if (auto_retry_count < 1) { + // First failure — schedule one silent retry before surfacing the error. + // Incrementing search_version triggers the $effect, which rebuilds qry_key + // with the new version and calls handle_search_refresh again. + auto_retry_count++; + console.log(`[Search] API failed — auto-retry ${auto_retry_count}/1 in 3s...`); + $idaa_sess.recovery_meetings.qry__status = 'loading'; + setTimeout(() => { + $idaa_sess.recovery_meetings.search_version++; + }, 3000); + } else { + // Both attempts failed — surface the error distinctly. + // IMPORTANT: do NOT show "No meetings found" here. A network/API failure + // is not the same as a genuinely empty result. Conflating the two was why + // the "no meetings found" bug went undiagnosed for ~1 year — staff and users + // saw what looked like an empty list and assumed no data, not a failure. + $idaa_sess.recovery_meetings.qry__status = 'error'; + event_id_li = []; + } } } } @@ -339,11 +425,41 @@ if (browser) { Searching... + {:else if $idaa_sess.recovery_meetings.qry__status === 'error'} +
+

Unable to load meetings. Please try again.

+ +
{:else}
No recovery meetings found matching your criteria.
+ {#if show_cache_reset_btn} + +
+

Still not seeing meetings? Your local cache may be out of date.

+ +
+ {/if} {/if} {/if}