docs: update native app specifications and TODO for Phase 5 completion
- Synchronize Functional Spec with new system handlers (Wallpaper, Power, Recording, Displays). - Update Automation Scripts with finalized AppleScript handlers for PowerPoint and Keynote. - Mark IDAA and Phase 5 automation tasks as completed in TODO.md.
This commit is contained in:
Scott Idem
4
TODO.md
4
TODO.md
@@ -14,8 +14,8 @@ This is a list of tasks to be completed before the next event/show/conference.
|
||||
- [x] **Exhibit Search:** Standardized Exhibitor and Lead Tracking reactive search. (Completed 2026-01-28)
|
||||
|
||||
2. **IDAA Module Hardening:**
|
||||
- [ ] Audit Jitsi meeting integration for connection stability.
|
||||
- [ ] Investigate reported "issues with IDAA pages".
|
||||
- [x] Audit Jitsi meeting integration for connection stability. (Completed 2026-01-30)
|
||||
- [x] Investigate reported "issues with IDAA pages" (Jitsi reports). (Completed 2026-01-30)
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -2,37 +2,46 @@
|
||||
|
||||
**Target OS:** macOS (Primary), Linux/Windows (Secondary)
|
||||
**Languages:** AppleScript (`osascript`), Bash (`sh`), Node.js `child_process`
|
||||
**Implementation:** `src/main/shell_handlers.ts` (Electron App)
|
||||
|
||||
## 1. Presentation Control (AppleScript)
|
||||
## 1. Presentation Launch & Control (AppleScript)
|
||||
To provide a seamless "Podium Experience," the app uses AppleScript to control third-party presentation software.
|
||||
|
||||
### 1.1 Microsoft PowerPoint
|
||||
* **Intent:** `powerpoint:start`
|
||||
* **Script:**
|
||||
* **Launch:** `powerpoint:launch`
|
||||
```applescript
|
||||
tell application "Microsoft PowerPoint"
|
||||
activate
|
||||
open (POSIX file "[FILE_PATH]")
|
||||
run slide show settings of active presentation
|
||||
delay 1
|
||||
run slide show of active presentation
|
||||
end tell
|
||||
```
|
||||
* **Intent:** `powerpoint:quit`
|
||||
* **Script:** `quit application "Microsoft PowerPoint" saving no`
|
||||
* **Control:**
|
||||
* **Next:** `tell application "Microsoft PowerPoint" to next slide of slide show view of active presentation`
|
||||
* **Prev:** `tell application "Microsoft PowerPoint" to previous slide of slide show view of active presentation`
|
||||
* **Start:** `tell application "Microsoft PowerPoint" to run slide show of active presentation`
|
||||
* **Stop:** `tell application "Microsoft PowerPoint" to stop slide show of active presentation`
|
||||
* **Quit:** `quit application "Microsoft PowerPoint" saving no`
|
||||
|
||||
### 1.2 Apple Keynote
|
||||
* **Intent:** `keynote:start`
|
||||
* **Script:**
|
||||
* **Launch:** `keynote:launch`
|
||||
```applescript
|
||||
tell application "Keynote"
|
||||
activate
|
||||
open (POSIX file "[FILE_PATH]")
|
||||
delay 1
|
||||
start (front document)
|
||||
end tell
|
||||
```
|
||||
* **Control:**
|
||||
* **Next:** `tell application "Keynote" to show next`
|
||||
* **Prev:** `tell application "Keynote" to show previous`
|
||||
* **Start:** `tell application "Keynote" to start (front document)`
|
||||
* **Stop:** `tell application "Keynote" to stop`
|
||||
|
||||
### 1.3 Adobe Acrobat (PDF)
|
||||
* **Intent:** `acrobat:fullscreen`
|
||||
* **Script:**
|
||||
* **Launch:** `acrobat:fullscreen`
|
||||
```applescript
|
||||
tell application "Adobe Acrobat Reader"
|
||||
activate
|
||||
@@ -42,6 +51,13 @@ To provide a seamless "Podium Experience," the app uses AppleScript to control t
|
||||
end tell
|
||||
```
|
||||
|
||||
### 1.4 LibreOffice (Linux Testing)
|
||||
* **Launch:** `libreoffice:launch`
|
||||
```bash
|
||||
libreoffice --impress "[FILE_PATH]"
|
||||
```
|
||||
* **Note:** LibreOffice control on Linux is currently limited to launch-only.
|
||||
|
||||
## 2. Process Management (Bash/Node)
|
||||
The app must ensure that only one presentation is "Active" to prevent audio overlap or confusion.
|
||||
|
||||
@@ -107,19 +123,19 @@ To keep the bridge secure, we do **not** send raw scripts from the UI. Instead,
|
||||
|
||||
**Renderer Call:**
|
||||
```javascript
|
||||
native_app.launcher.execIntent('presentation:start', { path: '/tmp/my_deck.pptx' });
|
||||
window.aetherNative.launch_presentation({ path: '/tmp/my_deck.pptx', app: 'powerpoint' });
|
||||
window.aetherNative.control_presentation({ app: 'powerpoint', action: 'next' });
|
||||
```
|
||||
|
||||
**Main Process Handler:**
|
||||
**Main Process Handler (`src/main/shell_handlers.ts`):**
|
||||
1. Verify the file path is within the allowed `[tmp]` directory.
|
||||
2. Identify the file extension (`.pptx`).
|
||||
3. Select the corresponding AppleScript template.
|
||||
4. Replace `[FILE_PATH]` and execute via `child_process.exec`.
|
||||
2. Select the corresponding AppleScript template based on `app` and `action`.
|
||||
3. Replace `[FILE_PATH]` and execute via `child_process.exec`.
|
||||
|
||||
## 7. Linux Development Mocking
|
||||
Since development happens on Arch Linux, the `execIntent` handler will check `process.platform`:
|
||||
- If `darwin`: Execute `osascript` or `sh`.
|
||||
- If `linux`: Write the intended command to a log file at `~/OSIT/automation_debug.log` and simulate a "Success" return.
|
||||
- If `darwin`: Execute `osascript`.
|
||||
- If `linux`: Use `libreoffice` for launch, or log intent for control commands.
|
||||
- **Recording Mock:** Use `ffmpeg` or generate a fake `.mp4` file to test the sync/upload logic.
|
||||
|
||||
## 8. Refresh & Sync Loops
|
||||
|
||||
@@ -1,53 +1,75 @@
|
||||
# Functional Specification: Aether Native App
|
||||
# Functional Specification: Aether Native App (Electron)
|
||||
|
||||
**Date:** 2026-01-20
|
||||
**Status:** Legacy Documentation for Rewrite
|
||||
**Source Files:** `aether_app_native/index.js`, `aether_app_native_v4.js`
|
||||
**Date:** 2026-01-30
|
||||
**Status:** V5 / Phase 5 Specification
|
||||
**Target OS:** macOS (Primary), Linux/Windows (Secondary)
|
||||
|
||||
## 1. Core Objectives
|
||||
The Native App acts as a high-privilege bridge for the Aether SvelteKit UI, providing:
|
||||
1. **Local File Orchestration:** Managing a local cache of presentation files.
|
||||
2. **Process Management:** Opening and closing third-party applications (PowerPoint, Keynote).
|
||||
3. **Hardware Integration:** Accessing device metadata and AV permissions.
|
||||
4. **Persistent Identity:** Maintaining a fixed device ID independent of browser cookies.
|
||||
The Native App acts as a "Privileged Shell" for the Aether SvelteKit UI, providing capabilities that standard web browsers block for security reasons. It turns a standard laptop into a managed "Podium Kiosk."
|
||||
|
||||
## 2. Detailed Function Mapping
|
||||
## 2. macOS Permission Matrix (TCC)
|
||||
To function as a podium controller, the Electron app requires specific entitlements.
|
||||
|
||||
### 2.1 Configuration Layer
|
||||
| Function | Process | Description | V3 Rewrite Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| `load_init_config` | Renderer | Reads `ae_native_app_sk_config.json` from disk. | Standardize path to `~/.config/aether/`. |
|
||||
| `get_url_cfg` | Renderer | Fetches remote config via API. | Switch from `/v2/crud/` to `/v3/crud/event_device/`. |
|
||||
| `import_config` | IPC | Passes config to Main process. | Use `contextBridge` for secure transfer. |
|
||||
| Permission | Reason | Criticality |
|
||||
| :--- | :--- | :--- |
|
||||
| **Accessibility** | Required to send AppleScript commands (Next/Prev) to Keynote and PowerPoint. | **CRITICAL** |
|
||||
| **Screen Recording** | Required by the `aperture` utility to record the presentation session. | **CRITICAL** |
|
||||
| **Microphone** | Required to capture room audio/presenter voice during recording. | **CRITICAL** |
|
||||
| **Camera** | Optional, only if "Presenter View" (PiP) recording is enabled. | Low |
|
||||
| **Automation** | Allows the app to control "System Events", "Finder", "Keynote", and "Microsoft PowerPoint". | **CRITICAL** |
|
||||
| **Full Disk Access** | Recommended to ensure access to `~/OSIT` and `~/Downloads` without nagging prompts. | High |
|
||||
|
||||
### 2.2 File & Cache System
|
||||
| Function | Process | Description | V3 Rewrite Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| `download_file` | IPC (Main) | Downloads via Axios stream to `.tmp` then renames. | Integrate JWT in download headers. |
|
||||
| `check_hash_cache`| Renderer | Checks shard subdirectories (e.g. `/cache/af/...`). | Keep sharding logic for performance. |
|
||||
| `open_hash_to_temp`| IPC (Main) | Maps `hash.file` -> `OriginalName.ext` in temp dir. | Ensure auto-cleanup of temp files on exit. |
|
||||
| `open_local_file` | IPC (Main) | Wrapper for Electron `shell.openPath`. | Restrict to whitelisted directories. |
|
||||
## 3. Handler Registry (V5)
|
||||
The following IPC handlers are exposed via `window.aetherNative`.
|
||||
|
||||
### 2.3 System Control
|
||||
| Function | Process | Description | V3 Rewrite Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| `kill_processes` | IPC (Main) | Uses `killall` (Mac) or `pkill` (Linux). | Add specialized support for Svelte 5 process tracking. |
|
||||
| `run_osascript` | IPC (Main) | Executes AppleScript (macOS only). | Add a "Mock Mode" for Linux development. |
|
||||
| `run_cmd` | IPC (Main) | Executes arbitrary shell commands. | **SECURITY:** Change to a whitelist of allowed commands. |
|
||||
### 3.1 File & Cache System
|
||||
| Handler | Description |
|
||||
| :--- | :--- |
|
||||
| `native:check-cache` | Verifies file existence in the sharded local cache (`~/OSIT/native_app/cache/xx/hash.file`). |
|
||||
| `native:download-to-cache` | Streams a file from the API to the local cache with SHA-256 verification. |
|
||||
| `native:launch-from-cache` | **Atomic Launch:** Copies file from cache -> temp with original filename -> launches app. |
|
||||
|
||||
### 2.4 Device & Environment
|
||||
| Function | Process | Description | V3 Rewrite Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| `get_device_info` | IPC (Main) | Returns OS, Arch, and Network info. | Use for V3 device heartbeat registration. |
|
||||
| `sys_perms` | Main | Requests Mac AV permissions. | Move to a dedicated bootstrap module. |
|
||||
### 3.2 Automation & Control (Phase 5)
|
||||
| Handler | Description |
|
||||
| :--- | :--- |
|
||||
| `native:launch-presentation` | Opens a file using the specific app (Keynote/PPT) with "Slide Show" mode enabled immediately. |
|
||||
| `native:control-presentation` | Sends navigation commands: `next`, `prev`, `start`, `stop`. |
|
||||
| `native:kill-processes` | Force quits applications by name (e.g., "Microsoft PowerPoint") to resolve freezes. |
|
||||
|
||||
## 3. The "Arch Linux vs macOS" Strategy
|
||||
To ensure the app remains developable on Arch Linux while targeting macOS:
|
||||
1. **Platform Detection:** Every system call must be gated by `process.platform`.
|
||||
2. **Path Placeholders:** Continue using `[home]` and `[tmp]` strings in config files, resolving them at runtime.
|
||||
3. **Mocking:** Create a `mock_native.js` that logs AppleScript/Terminal commands to the console instead of trying to execute them when `platform !== 'darwin'`.
|
||||
### 3.3 System Management (New Requirements)
|
||||
These handlers are required for full "Kiosk" management.
|
||||
|
||||
## 4. Security Requirements (Modernization)
|
||||
1. **Context Isolation:** Enable `contextIsolation: true`.
|
||||
2. **No Node Integration:** Disable `nodeIntegration` in the renderer.
|
||||
3. **IPC Whitelisting:** Replace `run_cmd` with specific named handlers (e.g., `native:open-powerpoint`) to prevent RCE (Remote Code Execution) vulnerabilities.
|
||||
| Handler | Description | Implementation Note |
|
||||
| :--- | :--- | :--- |
|
||||
| `native:set-wallpaper` | Resets desktop background to a specific image. | AppleScript: `tell application "System Events" to set picture of every desktop...` |
|
||||
| `native:update-app` | Checks `~/OSIT/admin_share/...` for newer versions and self-updates. | Requires specific logic to swap the `.app` bundle and restart. |
|
||||
| `native:window-control` | Controls the Electron window: `maximize`, `minimize`, `kiosk`, `devtools`. | Direct Electron `BrowserWindow` API calls. |
|
||||
| `native:manage-recording` | Controls `aperture` process: `start`, `stop`, `status`. | Must track the PID of the spawned recording process. |
|
||||
| `native:set-display-layout` | Toggles between **Mirror** and **Extended** modes. | Use bundled `displayplacer` binary (AppleScript is unreliable for this). |
|
||||
| `native:power-control` | Initiates `shutdown` or `restart`. | Command: `shutdown -h now` (may require admin/sudo setup or user-level fallback). |
|
||||
| `native:open-external` | Opens a URL in a specific browser (Chrome/Firefox). | `spawn('open', ['-a', 'Google Chrome', url])`. |
|
||||
|
||||
## 4. Feature Specifications
|
||||
|
||||
### 4.1 Self-Update Strategy
|
||||
1. **Path:** `~/OSIT/Speaker Ready System/Admin Share/Custom Applications/osit_binaries/`
|
||||
2. **Logic:**
|
||||
* Compare `package.json` version in running app vs. remote path.
|
||||
* If remote > local:
|
||||
* Display "Updating..." splash.
|
||||
* Copy new `.app` to `/Applications` (or current location).
|
||||
* Relaunch.
|
||||
|
||||
### 4.2 Web Page Handling
|
||||
The Launcher avoids Safari.
|
||||
* **Strategy A (Simple):** `native:open-external` -> Opens in Google Chrome Kiosk Mode (`--kiosk`).
|
||||
* **Strategy B (Integrated):** Use an Electron `BrowserView` (not `iframe`) to render the website *inside* the Aether window, ensuring we keep the "Back" button overlay visible.
|
||||
|
||||
### 4.3 Display Layouts
|
||||
To reliably switch between Mirroring (for setup) and Extended (for presentation):
|
||||
* **Tool:** Bundle **`displayplacer`** (Homebrew utility) inside `resources/bin`.
|
||||
* **Command:** `displayplacer "id:<main> res:1920x1080" "id:<proj> res:1920x1080 origin:(1920,0)"` (Example).
|
||||
|
||||
## 5. Security Guardrails
|
||||
* **IPC Whitelisting:** All `run-cmd` calls are deprecated in favor of specific named handlers (e.g., `set-wallpaper`) to prevent arbitrary command execution.
|
||||
* **Path Validation:** All file operations must be restricted to `~/OSIT` or `~/tmp` directories.
|
||||
|
||||
Reference in New Issue
Block a user