OSIT-AE-App-Svelte/documentation/NATIVE_APP_AUTOMATION_SCRIPTS.md

Specification: Aether Native Host Automation & Scripting

Target OS: macOS (Primary), Linux/Windows (Secondary) Languages: AppleScript (osascript), Bash (sh), Node.js child_process Implementation: src/main/shell_handlers.ts (Electron App)

1. Presentation Launch & Control (AppleScript)

To provide a seamless "Podium Experience," the app uses AppleScript to control third-party presentation software.

1.1 Microsoft PowerPoint

• Launch: powerpoint:launch

  tell application "Microsoft PowerPoint"
      activate
      open (POSIX file "[FILE_PATH]")
      delay 1
      run slide show of active presentation
  end tell
  

• Control:
  • Next: tell application "Microsoft PowerPoint" to next slide of slide show view of active presentation
  • Prev: tell application "Microsoft PowerPoint" to previous slide of slide show view of active presentation
  • Start: tell application "Microsoft PowerPoint" to run slide show of active presentation
  • Stop: tell application "Microsoft PowerPoint" to stop slide show of active presentation
• Quit: quit application "Microsoft PowerPoint" saving no

1.2 Apple Keynote

• Launch: keynote:launch

  tell application "Keynote"
      activate
      open (POSIX file "[FILE_PATH]")
      delay 1
      start (front document)
  end tell
  

• Control:
  • Next: tell application "Keynote" to show next
  • Prev: tell application "Keynote" to show previous
  • Start: tell application "Keynote" to start (front document)
  • Stop: tell application "Keynote" to stop

1.3 Adobe Acrobat (PDF)

• Launch: acrobat:fullscreen

  tell application "Adobe Acrobat Reader"
      activate
      -- Acrobat doesn't have a direct "Start Slideshow" AppleScript command
      -- We use system events to trigger Cmd+L (Full Screen)
      tell application "System Events" to keystroke "l" using {command down}
  end tell
  

1.4 LibreOffice (Linux Testing)

• Launch: libreoffice:launch

  libreoffice --impress "[FILE_PATH]"
  

• Note: LibreOffice control on Linux is currently limited to launch-only.

2. Process Management (Bash/Node)

The app must ensure that only one presentation is "Active" to prevent audio overlap or confusion.

Action	macOS Command	Linux/Windows Mock
Kill Office	killall -INT "Microsoft PowerPoint"	pkill -f powerpnt
Kill Keynote	killall -INT "Keynote"	echo "Keynote Not Supported"
Kill Acrobat	killall -9 "Acrobat Reader"	pkill -f acroread
Kill Parallels	pkill -i -f '(Parallels).*(PowerPoint)'	N/A

3. Recording Lifecycle (aperture_wrapper)

The app manages high-quality screen and audio recording using the aperture CLI or a custom wrapper.

3.1 Start Recording

• Intent: record:start
• Action: Launch aperture as a child process.
• Command Pattern:

  aperture --fps 30 --highlight-clicks --audio-device "[DEVICE_ID]" --output "[PATH]/[SESSION_ID].mp4"
  

• Telemetry: Push record_status: "Recording" and the PID to the V3 API.

3.2 Stop Recording

• Intent: record:stop
• Action:
  1. Send SIGINT (Interrupt) to the recorded process.
  2. Wait for file finalization.
  3. Verify file existence and minimum size.
  4. Push record_status: "Idle" to the V3 API.

4. Host Hardening & "Odd Stuff"

Specialized scripts to ensure the Mac laptop behaves as a dedicated podium.

4.1 Podium Lockdown (macOS)

• Prevent Sleep: caffeinate -dis & (Stores PID).
• Volume Enforce: osascript -e "set volume output volume 100"
• Timezone Sync: Force America/New_York (ET) for all device status pushes to ensure dashboard consistency.
• Hide Menu Bar/Dock: (Optional) Use System Events AppleScript to toggle auto-hide.

4.2 Digital Poster Screen Saver

To prevent screen burn-in and provide branding when the podium is idle:

• Idle Detection: Uses svelte-idle to track user activity.
• Action: Automatically rotates through a random list of event_file objects marked as "poster".
• Override: Any mouse movement or click immediately kills the screen saver modal.

4.3 WebSocket Remote Control (The "Tech Bridge")

Allows onsite staff to manage the podium without physical access.

• Intents:
  • ae_refresh:now: Triggers a full page reload.
  • ae_native:cmd: Executes a whitelisted shell command.
  • ae_open:event_file: Remotely opens a specific presentation or image.
  • ae_record:start/stop: Remotely toggles the Aperture wrapper.

5. Accessibility & Permissions (macOS TCC)

Since this is a specialized app, we must handle Apple's TCC (Transparency, Consent, and Control) layer:

• AV Access: Use systemPreferences.askForMediaAccess('camera') and microphone.
• Accessibility: For remote control/automation, check systemPreferences.isTrustedAccessibilityClient(true).
• Screen Recording: Triggered by the first aperture call; the app should provide a UI guide to help the user enable it in System Settings.
• Check Command: osascript -e 'tell application "System Events" to get UI elements enabled'

6. Automation Strategy (The "Intent" Pattern)

To keep the bridge secure, we do not send raw scripts from the UI. Instead, we send an "Intent."

Renderer Call:

window.aetherNative.launch_presentation({ path: '/tmp/my_deck.pptx', app: 'powerpoint' });
window.aetherNative.control_presentation({ app: 'powerpoint', action: 'next' });

Main Process Handler (src/main/shell_handlers.ts):

1. Verify the file path is within the allowed [tmp] directory.
2. Select the corresponding AppleScript template based on app and action.
3. Replace [FILE_PATH] and execute via child_process.exec.

7. Linux Development Mocking

Since development happens on Arch Linux, the execIntent handler will check process.platform:

• If darwin: Execute osascript.
• If linux: Use libreoffice for launch, or log intent for control commands.
• Recording Mock: Use ffmpeg or generate a fake .mp4 file to test the sync/upload logic.

8. Refresh & Sync Loops

Decoupled polling to balance performance and responsiveness:

• Session Loop (30s): High frequency for active podium changes.
• Location Loop (2m): Medium frequency for room configuration.
• Device Loop (5m): Low frequency for background heartbeat and telemetry.
• Sync Loop (15m): Background file warming for upcoming presentations.