fix(sw): complete cache-clearing + add controllerchange auto-reload

All cache-clearing buttons and the IDAA clear-caches page previously
cleared IDB/localStorage but left service worker registrations and Cache
Storage intact. On the next reload the SW re-served the old JS bundle,
leaving users stuck on stale code despite appearing to reload. This
caused recurring stale-state reports from IDAA and other clients for
4+ months.

Two gaps closed:
1. Every clear path (root page buttons, sys bar, help tech, idaa/clear-caches)
   now unregisters SW registrations and clears Cache Storage before touching
   IDB and localStorage. Order: SW → Cache Storage → IDB → localStorage.
2. Added controllerchange listener in +layout.svelte effect 4. When the new
   SW activates and calls clients.claim(), this listener reloads the page so
   open tabs run the new JS bundle instead of keeping old code in memory
   indefinitely. Without this, skipWaiting + clients.claim work correctly on
   the SW side but the page side never picks up the update.

Also added thorough code comments and updated REFERENCE__Common_Agent_Mistakes
(#15) and BOOTSTRAP doc (#8) to document the full root cause so this cannot
be silently re-broken by a future agent or refactor.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

documentation/BOOTSTRAP__AI_Agent_Quickstart.md

@@ -299,7 +299,11 @@ Read this section first, then open the reference doc when your task touches one
 5. **Sorting and search correctness** — `tmp_sort_*` comparator direction and Dexie sorting caveats.
 6. **Network reliability** — retry classification in `api_*_object.ts` and timeout behavior.
 7. **JSON field safety** — `*_json` null reads/writes and wrapper serialization behavior.
-8. **Service worker rollout behavior** — stale-tab symptoms, activation expectations, and trade-offs.
+8. **Service worker + cache clearing — MANDATORY four-layer wipe** — clearing IDB/localStorage
+   alone is a placebo. The SW Cache Storage is separate and must be cleared first or the SW
+   re-serves the old JS bundle on the very next reload. The `controllerchange` event listener in
+   `+layout.svelte` (effect 4) is also required or open tabs stay on old code after deploys.
+   See mistake #15 in the reference doc — this caused a recurring client issue for 4+ months.
 9. **Local/remote config sync** — shadow fields that silently bypass an admin-synced master
    field, SWR-await-after-write races, and stateful/conditional sync gates that desync local
    state from history rather than current config. See the Pres Mgmt Config sync overhaul

documentation/REFERENCE__Common_Agent_Mistakes.md

@@ -141,14 +141,55 @@ Review policy: balanced curation.
 
 **Verify:** Audit reads/writes for `cfg_json`, `data_json`, `poc_kv_json`, and similar nullable blob fields.
 
-### 15) Service worker stale-tab behavior misunderstood
-**Impact:** users run old code longer than expected, “can’t reproduce” bug reports.
+### 15) Incomplete cache clearing — SW and Cache Storage omitted
 
-**What happened:** deployment assumptions ignored SW activation lifecycle.
+**Impact:** Sev-1 class. Users stuck on old JS for weeks after deployments. “Fixed” bugs
+reappear because the affected user never loaded the fix. Nearly cost the IDAA client after
+4+ months of recurring reports that were repeatedly declared resolved.
 
-**Rule:** Keep SW activation behavior explicit (`skipWaiting`, `clients.claim`) and evaluate trade-offs for session-heavy flows.
+**What happened (two separate gaps, both required to fix):**
 
-**Verify:** After deploy, validate that long-lived tabs pick up new SW behavior as intended.
+**Gap A — Clear buttons did not clear the SW or Cache Storage.**
+The app has a service worker that caches every JS/CSS/HTML asset in its own Cache Storage
+(separate from IndexedDB and localStorage). All “clear caches” buttons only cleared IDB,
+localStorage, and sessionStorage — leaving the SW and its Cache Storage intact. On the next
+reload, the SW intercepted the request and returned the OLD cached JS bundle. The user appeared
+to reload but was still running old code. Clicking “Clear Storage and Reload” appeared to work
+but did nothing to break the stuck state.
+
+**Gap B — No `controllerchange` listener in the layout.**
+`service-worker.js` correctly calls `skipWaiting()` on install (new SW activates immediately)
+and `clients.claim()` on activate (new SW takes control of all open tabs). But the page-side
+half of this contract was missing: there was no `controllerchange` event listener to trigger
+a reload when the new SW took over. Result: every deployment correctly installed and activated
+a new SW, but any tab that was already open kept running the old JS bundle from memory —
+indefinitely, until the user happened to close and reopen the browser.
+
+**Who was affected:** Windows users who leave Chrome/Edge running for days or weeks with the
+tab pinned or restored via “continue where you left off.” Most users were never affected
+because normal browser habits (restart, close/reopen tab) naturally clear the stuck state.
+The pattern looked random but was actually a long-session-duration problem.
+
+**The fix (2026-06-22):**
+1. Every “clear” button and automated heal path now clears SW registrations + Cache Storage
+   BEFORE clearing IDB/localStorage. Order matters: SW → Cache Storage → IDB → storage.
+2. `controllerchange` listener added to `+layout.svelte` effect 4 — reloads the page when
+   a new SW takes control, so open tabs automatically get fresh JS after any deployment.
+3. `idaa/clear-caches/+page.svelte` (IDAA iframe tool) updated with the same fix.
+
+**Rule:** Any code that clears caches MUST clear all four layers in this order:
+1. `navigator.serviceWorker.getRegistrations()` → unregister each
+2. `caches.keys()` → delete each key
+3. `indexedDB.databases()` → delete each database
+4. `localStorage.clear()` + `sessionStorage.clear()`
+
+Clearing IDB/localStorage WITHOUT clearing SW and Cache Storage is not a fix — it is a
+placebo that makes the user think something changed while the old JS bundle is served again
+on the very next reload.
+
+**Verify:** After any future changes to cache-clearing paths, confirm all four layers are
+cleared. The `controllerchange` listener in `+layout.svelte` effect 4 must not be removed.
+`/testing/fix-sw` is the emergency escape route for users who are stuck on pre-fix code.
 
 ### 16) Local "shadow field" silently bypasses the admin-synced master field
 **Impact:** an admin config toggle appears to do nothing — the UI updates fine when a local

src/lib/app_components/e_app_help_tech.svelte

@@ -459,11 +459,18 @@ class:to-90%={$ae_sess.show_help_tech} -->
                     onclick={async () => {
                         const edit_mode = $ae_loc.edit_mode;
                         const confirm_msg = edit_mode
-                            ? 'Clear all IDB caches, localStorage, and sessionStorage? Your sign-in will be preserved. This will reload the page.'
-                            : 'Clear all IDB caches? This will reload the page.';
+                            ? 'Clear all IDB caches, service worker caches, localStorage, and sessionStorage? Your sign-in will be preserved. This will reload the page.'
+                            : 'Clear all IDB caches and service worker caches? This will reload the page.';
 
                         if (!confirm(confirm_msg)) return;
 
+                        if ('serviceWorker' in navigator) {
+                            const registrations = await navigator.serviceWorker.getRegistrations();
+                            for (const reg of registrations) await reg.unregister();
+                        }
+                        const cache_keys = await caches.keys();
+                        for (const key of cache_keys) await caches.delete(key);
+
                         // Enumerate and delete every IDB database on this origin.
                         const db_list = await indexedDB.databases();
                         console.log('[clear_reload] IDB databases found:', db_list.map((d) => d.name));
@@ -501,12 +508,19 @@ class:to-90%={$ae_sess.show_help_tech} -->
                     onclick={async () => {
                         if (
                             !confirm(
-                                'FULL RESET: Delete ALL IndexedDB databases, clear localStorage and sessionStorage, then reload? This cannot be undone.'
+                                'FULL RESET: Unregister service workers, clear all caches, delete ALL IndexedDB databases, clear localStorage and sessionStorage, then reload? This cannot be undone.'
                             )
                         ) {
                             return;
                         }
 
+                        if ('serviceWorker' in navigator) {
+                            const registrations = await navigator.serviceWorker.getRegistrations();
+                            for (const reg of registrations) await reg.unregister();
+                        }
+                        const cache_keys = await caches.keys();
+                        for (const key of cache_keys) await caches.delete(key);
+
                         // Enumerate every IDB database on this origin and delete them all.
                         const db_list = await indexedDB.databases();
                         console.log('[clear_all] IDB databases found:', db_list.map((d) => d.name));

src/lib/app_components/e_app_sys_bar.svelte

@@ -99,14 +99,25 @@ async function handle_clear_idb_only() {
     window.location.reload();
 }
 
-// ── Dev: clear all browser storage + all IndexedDB databases, then reload ──
+// ── Dev: full reset — SW + Cache Storage + IDB + localStorage/sessionStorage ──
+// SW and Cache Storage MUST be cleared here. Clearing IDB/localStorage alone leaves
+// the SW serving old JS bundles from its own Cache Storage on the next reload,
+// which means the user stays stuck on old code. Order: SW → cache → IDB → storage.
 async function handle_clear_storage_and_idb() {
     if (
         !confirm(
-            'FULL RESET: Delete ALL IndexedDB databases, clear localStorage and sessionStorage, then reload? This cannot be undone.'
+            'FULL RESET: Unregister service workers, clear all caches, delete IndexedDB databases, clear localStorage and sessionStorage, then reload? This cannot be undone.'
         )
     )
         return;
+
+    if ('serviceWorker' in navigator) {
+        const registrations = await navigator.serviceWorker.getRegistrations();
+        for (const reg of registrations) await reg.unregister();
+    }
+    const cache_keys = await caches.keys();
+    for (const key of cache_keys) await caches.delete(key);
+
     const db_list = await indexedDB.databases();
     console.log('[clear_all] IDB databases found:', db_list.map((d) => d.name));
     for (const db of db_list) {

src/routes/+layout.svelte

@@ -274,6 +274,27 @@ function clear_sess() {
     sessionStorage.clear();
 }
 
+// CRITICAL — all three storage layers must be cleared together or the fix is incomplete.
+//
+// The service worker (SW) maintains its own Cache Storage that is entirely separate from
+// IndexedDB, localStorage, and sessionStorage. The SW uses Cache Storage to serve the
+// app's JS/CSS bundles and HTML shell directly from disk — bypassing the network entirely
+// for those assets. This means:
+//
+//   Clearing IDB + localStorage ONLY:
+//     → SW is still registered and still serving old JS bundles from its cache.
+//     → Next reload: SW intercepts, returns OLD cached JS → user is still stuck.
+//
+//   Clearing SW registrations ONLY:
+//     → Cache Storage still contains old entries. A newly registered SW may serve them
+//       before it rebuilds the cache from the network, causing a transient stale state.
+//
+//   Clearing BOTH SW registrations AND Cache Storage, THEN IDB/localStorage:
+//     → Next reload: no SW intercepts → fresh HTML from server → fresh JS bundle
+//       filenames → fresh JS loaded → user is on current code. ✅
+//
+// This function is called by the version-mismatch detection path (flag_hard_reload).
+// All manual cache-clearing buttons in the app must follow the same order.
 async function clear_stale_service_worker_state() {
     if (!browser || sw_heal_in_flight) return;
     sw_heal_in_flight = true;
@@ -307,6 +328,27 @@ async function clear_stale_service_worker_state() {
 $effect(() => {
     if (!browser) return;
 
+    // SW update → page reload bridge. DO NOT REMOVE.
+    //
+    // How SW updates propagate (all three steps are required):
+    //   1. service-worker.js calls skipWaiting() on install → new SW activates immediately
+    //      instead of waiting for all existing tabs to close.
+    //   2. service-worker.js calls clients.claim() on activate → new SW takes control of
+    //      all currently open tabs without requiring a navigation.
+    //   3. THIS LISTENER: when the new SW takes control, `controllerchange` fires on every
+    //      open tab. We reload so the tab runs the new JS bundle rather than keeping the
+    //      old JS that was already loaded in memory.
+    //
+    // Without step 3, users can be stuck on old code indefinitely after a deployment —
+    // even though the SW correctly updated under them. The tab shows no error; it just
+    // silently runs whatever JS it had at the time the SW changed. This caused IDAA and
+    // other users to run stale code for weeks at a time across multiple deployments.
+    // Added 2026-06-22 to close this gap.
+    const on_controller_change = () => window.location.reload();
+    if ('serviceWorker' in navigator) {
+        navigator.serviceWorker.addEventListener('controllerchange', on_controller_change);
+    }
+
     // Initialise PWA install prompt listener (captures beforeinstallprompt early).
     pwa_install.init();
 
@@ -390,7 +432,12 @@ $effect(() => {
         if (!$ae_loc.is_native) $ae_loc.is_native = true;
     }
 
-    return () => window.removeEventListener('message', handler);
+    return () => {
+        window.removeEventListener('message', handler);
+        if ('serviceWorker' in navigator) {
+            navigator.serviceWorker.removeEventListener('controllerchange', on_controller_change);
+        }
+    };
 });
 
 // 5. SESSION EXPIRED EFFECT — watches ae_auth_error and raises the banner when the API signals 401/403.

src/routes/+page.svelte

@@ -91,11 +91,18 @@ onMount(() => {
                 onclick={async () => {
                     const edit_mode = $ae_loc.edit_mode;
                     const confirm_msg = edit_mode
-                        ? 'Clear all IDB caches, localStorage, and sessionStorage? Your sign-in will be preserved. This will reload the page.'
-                        : 'Clear all IDB caches? This will reload the page.';
+                        ? 'Clear all IDB caches, service worker caches, localStorage, and sessionStorage? Your sign-in will be preserved. This will reload the page.'
+                        : 'Clear all IDB caches and service worker caches? This will reload the page.';
 
                     if (!confirm(confirm_msg)) return;
 
+                    if ('serviceWorker' in navigator) {
+                        const registrations = await navigator.serviceWorker.getRegistrations();
+                        for (const reg of registrations) await reg.unregister();
+                    }
+                    const cache_keys = await caches.keys();
+                    for (const key of cache_keys) await caches.delete(key);
+
                     const db_list = await indexedDB.databases();
                     for (const db of db_list) {
                         if (db.name) indexedDB.deleteDatabase(db.name);
@@ -111,7 +118,7 @@ onMount(() => {
                     window.location.reload();
                 }}
                 class="btn btn-sm preset-tonal-surface hover:preset-outlined-warning text-error-300 hover:text-error-800 m-1 transition-all"
-                title="Clear & Reload: Delete all IDB caches and reload. In edit mode also clears localStorage/sessionStorage, preserving your sign-in.">
+                title="Clear & Reload: Delete all IDB caches, unregister service workers, and reload. In edit mode also clears localStorage/sessionStorage, preserving your sign-in.">
                 <!-- <span class="fas fa-sync mx-1"></span> -->
                 <RefreshCw class="mx-1" />
                 Reload
@@ -121,12 +128,21 @@ onMount(() => {
                 onclick={async () => {
                     if (
                         !confirm(
-                            'FULL RESET: Delete ALL IndexedDB databases, clear localStorage and sessionStorage, then reload? This cannot be undone.'
+                            'FULL RESET: Unregister service workers, clear all caches, delete IndexedDB databases, clear localStorage and sessionStorage, then reload? This cannot be undone.'
                         )
                     ) {
                         return;
                     }
 
+                    // Unregister service workers and clear Cache Storage so stale
+                    // SW-cached assets don't survive the reset and re-serve old bundles.
+                    if ('serviceWorker' in navigator) {
+                        const registrations = await navigator.serviceWorker.getRegistrations();
+                        for (const reg of registrations) await reg.unregister();
+                    }
+                    const cache_keys = await caches.keys();
+                    for (const key of cache_keys) await caches.delete(key);
+
                     const db_list = await indexedDB.databases();
                     console.log('[clear_all] IDB databases found:', db_list.map((d) => d.name));
                     for (const db of db_list) {
@@ -139,7 +155,7 @@ onMount(() => {
                     window.location.reload();
                 }}
                 class="btn btn-sm preset-tonal-surface hover:preset-outlined-warning text-error-300 hover:text-error-800 m-1 p-1 transition-all"
-                title="Full Reset: Delete ALL IndexedDB databases, clear localStorage and sessionStorage for this origin, then reload.">
+                title="Full Reset: Unregister service workers, clear all caches, delete IndexedDB databases, clear localStorage and sessionStorage for this origin, then reload.">
                 <!-- <span class="fas fa-sync mx-1"></span> -->
                 <RefreshCcwDot class="mx-1" />
                 Clear Storage and Reload

src/routes/idaa/clear-caches/+page.svelte

@@ -11,6 +11,8 @@ interface ClearStep {
 }
 
 let steps: ClearStep[] = $state([
+    { label: 'Service workers', status: 'pending', detail: '' },
+    { label: 'Service worker caches', status: 'pending', detail: '' },
     { label: 'IndexedDB databases', status: 'pending', detail: '' },
     { label: 'Local storage', status: 'pending', detail: '' },
     { label: 'Session storage', status: 'pending', detail: '' }
@@ -20,45 +22,79 @@ let overall_done = $state(false);
 let had_error = $state(false);
 
 async function clear_all_caches() {
-    // IDB — enumerate and delete every database on this origin
+    // Service workers
     steps[0].status = 'running';
     try {
-        const db_list = await indexedDB.databases();
-        for (const db of db_list) {
-            if (db.name) indexedDB.deleteDatabase(db.name);
+        if ('serviceWorker' in navigator) {
+            const registrations = await navigator.serviceWorker.getRegistrations();
+            for (const reg of registrations) await reg.unregister();
+            steps[0].detail = `${registrations.length} unregistered`;
+        } else {
+            steps[0].detail = 'Not supported';
         }
         steps[0].status = 'done';
-        steps[0].detail = `${db_list.length} database${db_list.length !== 1 ? 's' : ''} cleared`;
     } catch (e) {
         steps[0].status = 'error';
         steps[0].detail = String(e);
         had_error = true;
     }
 
-    // localStorage
+    // Cache Storage (SW asset caches)
     steps[1].status = 'running';
     try {
-        localStorage.clear();
+        if ('caches' in window) {
+            const cache_keys = await caches.keys();
+            for (const key of cache_keys) await caches.delete(key);
+            steps[1].detail = `${cache_keys.length} cache${cache_keys.length !== 1 ? 's' : ''} cleared`;
+        } else {
+            steps[1].detail = 'Not supported';
+        }
         steps[1].status = 'done';
-        steps[1].detail = 'Cleared';
     } catch (e) {
         steps[1].status = 'error';
         steps[1].detail = String(e);
         had_error = true;
     }
 
-    // sessionStorage
+    // IDB — enumerate and delete every database on this origin
     steps[2].status = 'running';
     try {
-        sessionStorage.clear();
+        const db_list = await indexedDB.databases();
+        for (const db of db_list) {
+            if (db.name) indexedDB.deleteDatabase(db.name);
+        }
         steps[2].status = 'done';
-        steps[2].detail = 'Cleared';
+        steps[2].detail = `${db_list.length} database${db_list.length !== 1 ? 's' : ''} cleared`;
     } catch (e) {
         steps[2].status = 'error';
         steps[2].detail = String(e);
         had_error = true;
     }
 
+    // localStorage
+    steps[3].status = 'running';
+    try {
+        localStorage.clear();
+        steps[3].status = 'done';
+        steps[3].detail = 'Cleared';
+    } catch (e) {
+        steps[3].status = 'error';
+        steps[3].detail = String(e);
+        had_error = true;
+    }
+
+    // sessionStorage
+    steps[4].status = 'running';
+    try {
+        sessionStorage.clear();
+        steps[4].status = 'done';
+        steps[4].detail = 'Cleared';
+    } catch (e) {
+        steps[4].status = 'error';
+        steps[4].detail = String(e);
+        had_error = true;
+    }
+
     overall_done = true;
 
     // Notify parent window (Novi page) that the clear is complete.