From 7a8871c51fa72afee36a9a34582bb0713ede576e Mon Sep 17 00:00:00 2001 From: Scott Idem Date: Fri, 30 Jan 2026 11:35:01 -0500 Subject: [PATCH] docs: update native app specifications and TODO for Phase 5 completion - Synchronize Functional Spec with new system handlers (Wallpaper, Power, Recording, Displays). - Update Automation Scripts with finalized AppleScript handlers for PowerPoint and Keynote. - Mark IDAA and Phase 5 automation tasks as completed in TODO.md. --- TODO.md | 4 +- .../NATIVE_APP_AUTOMATION_SCRIPTS.md | 50 +++++--- documentation/NATIVE_APP_FUNCTIONAL_SPEC.md | 108 +++++++++++------- 3 files changed, 100 insertions(+), 62 deletions(-) diff --git a/TODO.md b/TODO.md index f1cd6be4..a6604344 100644 --- a/TODO.md +++ b/TODO.md @@ -14,8 +14,8 @@ This is a list of tasks to be completed before the next event/show/conference. - [x] **Exhibit Search:** Standardized Exhibitor and Lead Tracking reactive search. (Completed 2026-01-28) 2. **IDAA Module Hardening:** - - [ ] Audit Jitsi meeting integration for connection stability. - - [ ] Investigate reported "issues with IDAA pages". + - [x] Audit Jitsi meeting integration for connection stability. (Completed 2026-01-30) + - [x] Investigate reported "issues with IDAA pages" (Jitsi reports). (Completed 2026-01-30) --- diff --git a/documentation/NATIVE_APP_AUTOMATION_SCRIPTS.md b/documentation/NATIVE_APP_AUTOMATION_SCRIPTS.md index 59df7c97..75fd071b 100644 --- a/documentation/NATIVE_APP_AUTOMATION_SCRIPTS.md +++ b/documentation/NATIVE_APP_AUTOMATION_SCRIPTS.md @@ -2,37 +2,46 @@ **Target OS:** macOS (Primary), Linux/Windows (Secondary) **Languages:** AppleScript (`osascript`), Bash (`sh`), Node.js `child_process` +**Implementation:** `src/main/shell_handlers.ts` (Electron App) -## 1. Presentation Control (AppleScript) +## 1. Presentation Launch & Control (AppleScript) To provide a seamless "Podium Experience," the app uses AppleScript to control third-party presentation software. ### 1.1 Microsoft PowerPoint -* **Intent:** `powerpoint:start` -* **Script:** +* **Launch:** `powerpoint:launch` ```applescript tell application "Microsoft PowerPoint" activate open (POSIX file "[FILE_PATH]") - run slide show settings of active presentation + delay 1 + run slide show of active presentation end tell ``` -* **Intent:** `powerpoint:quit` -* **Script:** `quit application "Microsoft PowerPoint" saving no` +* **Control:** + * **Next:** `tell application "Microsoft PowerPoint" to next slide of slide show view of active presentation` + * **Prev:** `tell application "Microsoft PowerPoint" to previous slide of slide show view of active presentation` + * **Start:** `tell application "Microsoft PowerPoint" to run slide show of active presentation` + * **Stop:** `tell application "Microsoft PowerPoint" to stop slide show of active presentation` +* **Quit:** `quit application "Microsoft PowerPoint" saving no` ### 1.2 Apple Keynote -* **Intent:** `keynote:start` -* **Script:** +* **Launch:** `keynote:launch` ```applescript tell application "Keynote" activate open (POSIX file "[FILE_PATH]") + delay 1 start (front document) end tell ``` +* **Control:** + * **Next:** `tell application "Keynote" to show next` + * **Prev:** `tell application "Keynote" to show previous` + * **Start:** `tell application "Keynote" to start (front document)` + * **Stop:** `tell application "Keynote" to stop` ### 1.3 Adobe Acrobat (PDF) -* **Intent:** `acrobat:fullscreen` -* **Script:** +* **Launch:** `acrobat:fullscreen` ```applescript tell application "Adobe Acrobat Reader" activate @@ -42,6 +51,13 @@ To provide a seamless "Podium Experience," the app uses AppleScript to control t end tell ``` +### 1.4 LibreOffice (Linux Testing) +* **Launch:** `libreoffice:launch` + ```bash + libreoffice --impress "[FILE_PATH]" + ``` +* **Note:** LibreOffice control on Linux is currently limited to launch-only. + ## 2. Process Management (Bash/Node) The app must ensure that only one presentation is "Active" to prevent audio overlap or confusion. @@ -107,19 +123,19 @@ To keep the bridge secure, we do **not** send raw scripts from the UI. Instead, **Renderer Call:** ```javascript -native_app.launcher.execIntent('presentation:start', { path: '/tmp/my_deck.pptx' }); +window.aetherNative.launch_presentation({ path: '/tmp/my_deck.pptx', app: 'powerpoint' }); +window.aetherNative.control_presentation({ app: 'powerpoint', action: 'next' }); ``` -**Main Process Handler:** +**Main Process Handler (`src/main/shell_handlers.ts`):** 1. Verify the file path is within the allowed `[tmp]` directory. -2. Identify the file extension (`.pptx`). -3. Select the corresponding AppleScript template. -4. Replace `[FILE_PATH]` and execute via `child_process.exec`. +2. Select the corresponding AppleScript template based on `app` and `action`. +3. Replace `[FILE_PATH]` and execute via `child_process.exec`. ## 7. Linux Development Mocking Since development happens on Arch Linux, the `execIntent` handler will check `process.platform`: -- If `darwin`: Execute `osascript` or `sh`. -- If `linux`: Write the intended command to a log file at `~/OSIT/automation_debug.log` and simulate a "Success" return. +- If `darwin`: Execute `osascript`. +- If `linux`: Use `libreoffice` for launch, or log intent for control commands. - **Recording Mock:** Use `ffmpeg` or generate a fake `.mp4` file to test the sync/upload logic. ## 8. Refresh & Sync Loops diff --git a/documentation/NATIVE_APP_FUNCTIONAL_SPEC.md b/documentation/NATIVE_APP_FUNCTIONAL_SPEC.md index f2a158e3..ae3a793b 100644 --- a/documentation/NATIVE_APP_FUNCTIONAL_SPEC.md +++ b/documentation/NATIVE_APP_FUNCTIONAL_SPEC.md @@ -1,53 +1,75 @@ -# Functional Specification: Aether Native App +# Functional Specification: Aether Native App (Electron) -**Date:** 2026-01-20 -**Status:** Legacy Documentation for Rewrite -**Source Files:** `aether_app_native/index.js`, `aether_app_native_v4.js` +**Date:** 2026-01-30 +**Status:** V5 / Phase 5 Specification +**Target OS:** macOS (Primary), Linux/Windows (Secondary) ## 1. Core Objectives -The Native App acts as a high-privilege bridge for the Aether SvelteKit UI, providing: -1. **Local File Orchestration:** Managing a local cache of presentation files. -2. **Process Management:** Opening and closing third-party applications (PowerPoint, Keynote). -3. **Hardware Integration:** Accessing device metadata and AV permissions. -4. **Persistent Identity:** Maintaining a fixed device ID independent of browser cookies. +The Native App acts as a "Privileged Shell" for the Aether SvelteKit UI, providing capabilities that standard web browsers block for security reasons. It turns a standard laptop into a managed "Podium Kiosk." -## 2. Detailed Function Mapping +## 2. macOS Permission Matrix (TCC) +To function as a podium controller, the Electron app requires specific entitlements. -### 2.1 Configuration Layer -| Function | Process | Description | V3 Rewrite Notes | -| --- | --- | --- | --- | -| `load_init_config` | Renderer | Reads `ae_native_app_sk_config.json` from disk. | Standardize path to `~/.config/aether/`. | -| `get_url_cfg` | Renderer | Fetches remote config via API. | Switch from `/v2/crud/` to `/v3/crud/event_device/`. | -| `import_config` | IPC | Passes config to Main process. | Use `contextBridge` for secure transfer. | +| Permission | Reason | Criticality | +| :--- | :--- | :--- | +| **Accessibility** | Required to send AppleScript commands (Next/Prev) to Keynote and PowerPoint. | **CRITICAL** | +| **Screen Recording** | Required by the `aperture` utility to record the presentation session. | **CRITICAL** | +| **Microphone** | Required to capture room audio/presenter voice during recording. | **CRITICAL** | +| **Camera** | Optional, only if "Presenter View" (PiP) recording is enabled. | Low | +| **Automation** | Allows the app to control "System Events", "Finder", "Keynote", and "Microsoft PowerPoint". | **CRITICAL** | +| **Full Disk Access** | Recommended to ensure access to `~/OSIT` and `~/Downloads` without nagging prompts. | High | -### 2.2 File & Cache System -| Function | Process | Description | V3 Rewrite Notes | -| --- | --- | --- | --- | -| `download_file` | IPC (Main) | Downloads via Axios stream to `.tmp` then renames. | Integrate JWT in download headers. | -| `check_hash_cache`| Renderer | Checks shard subdirectories (e.g. `/cache/af/...`). | Keep sharding logic for performance. | -| `open_hash_to_temp`| IPC (Main) | Maps `hash.file` -> `OriginalName.ext` in temp dir. | Ensure auto-cleanup of temp files on exit. | -| `open_local_file` | IPC (Main) | Wrapper for Electron `shell.openPath`. | Restrict to whitelisted directories. | +## 3. Handler Registry (V5) +The following IPC handlers are exposed via `window.aetherNative`. -### 2.3 System Control -| Function | Process | Description | V3 Rewrite Notes | -| --- | --- | --- | --- | -| `kill_processes` | IPC (Main) | Uses `killall` (Mac) or `pkill` (Linux). | Add specialized support for Svelte 5 process tracking. | -| `run_osascript` | IPC (Main) | Executes AppleScript (macOS only). | Add a "Mock Mode" for Linux development. | -| `run_cmd` | IPC (Main) | Executes arbitrary shell commands. | **SECURITY:** Change to a whitelist of allowed commands. | +### 3.1 File & Cache System +| Handler | Description | +| :--- | :--- | +| `native:check-cache` | Verifies file existence in the sharded local cache (`~/OSIT/native_app/cache/xx/hash.file`). | +| `native:download-to-cache` | Streams a file from the API to the local cache with SHA-256 verification. | +| `native:launch-from-cache` | **Atomic Launch:** Copies file from cache -> temp with original filename -> launches app. | -### 2.4 Device & Environment -| Function | Process | Description | V3 Rewrite Notes | -| --- | --- | --- | --- | -| `get_device_info` | IPC (Main) | Returns OS, Arch, and Network info. | Use for V3 device heartbeat registration. | -| `sys_perms` | Main | Requests Mac AV permissions. | Move to a dedicated bootstrap module. | +### 3.2 Automation & Control (Phase 5) +| Handler | Description | +| :--- | :--- | +| `native:launch-presentation` | Opens a file using the specific app (Keynote/PPT) with "Slide Show" mode enabled immediately. | +| `native:control-presentation` | Sends navigation commands: `next`, `prev`, `start`, `stop`. | +| `native:kill-processes` | Force quits applications by name (e.g., "Microsoft PowerPoint") to resolve freezes. | -## 3. The "Arch Linux vs macOS" Strategy -To ensure the app remains developable on Arch Linux while targeting macOS: -1. **Platform Detection:** Every system call must be gated by `process.platform`. -2. **Path Placeholders:** Continue using `[home]` and `[tmp]` strings in config files, resolving them at runtime. -3. **Mocking:** Create a `mock_native.js` that logs AppleScript/Terminal commands to the console instead of trying to execute them when `platform !== 'darwin'`. +### 3.3 System Management (New Requirements) +These handlers are required for full "Kiosk" management. -## 4. Security Requirements (Modernization) -1. **Context Isolation:** Enable `contextIsolation: true`. -2. **No Node Integration:** Disable `nodeIntegration` in the renderer. -3. **IPC Whitelisting:** Replace `run_cmd` with specific named handlers (e.g., `native:open-powerpoint`) to prevent RCE (Remote Code Execution) vulnerabilities. +| Handler | Description | Implementation Note | +| :--- | :--- | :--- | +| `native:set-wallpaper` | Resets desktop background to a specific image. | AppleScript: `tell application "System Events" to set picture of every desktop...` | +| `native:update-app` | Checks `~/OSIT/admin_share/...` for newer versions and self-updates. | Requires specific logic to swap the `.app` bundle and restart. | +| `native:window-control` | Controls the Electron window: `maximize`, `minimize`, `kiosk`, `devtools`. | Direct Electron `BrowserWindow` API calls. | +| `native:manage-recording` | Controls `aperture` process: `start`, `stop`, `status`. | Must track the PID of the spawned recording process. | +| `native:set-display-layout` | Toggles between **Mirror** and **Extended** modes. | Use bundled `displayplacer` binary (AppleScript is unreliable for this). | +| `native:power-control` | Initiates `shutdown` or `restart`. | Command: `shutdown -h now` (may require admin/sudo setup or user-level fallback). | +| `native:open-external` | Opens a URL in a specific browser (Chrome/Firefox). | `spawn('open', ['-a', 'Google Chrome', url])`. | + +## 4. Feature Specifications + +### 4.1 Self-Update Strategy +1. **Path:** `~/OSIT/Speaker Ready System/Admin Share/Custom Applications/osit_binaries/` +2. **Logic:** + * Compare `package.json` version in running app vs. remote path. + * If remote > local: + * Display "Updating..." splash. + * Copy new `.app` to `/Applications` (or current location). + * Relaunch. + +### 4.2 Web Page Handling +The Launcher avoids Safari. +* **Strategy A (Simple):** `native:open-external` -> Opens in Google Chrome Kiosk Mode (`--kiosk`). +* **Strategy B (Integrated):** Use an Electron `BrowserView` (not `iframe`) to render the website *inside* the Aether window, ensuring we keep the "Back" button overlay visible. + +### 4.3 Display Layouts +To reliably switch between Mirroring (for setup) and Extended (for presentation): +* **Tool:** Bundle **`displayplacer`** (Homebrew utility) inside `resources/bin`. +* **Command:** `displayplacer "id:
res:1920x1080" "id: res:1920x1080 origin:(1920,0)"` (Example). + +## 5. Security Guardrails +* **IPC Whitelisting:** All `run-cmd` calls are deprecated in favor of specific named handlers (e.g., `set-wallpaper`) to prevent arbitrary command execution. +* **Path Validation:** All file operations must be restricted to `~/OSIT` or `~/tmp` directories.