Commit Graph

155 Commits

Author SHA1 Message Date
Scott Idem
6c6de37419 fix: restrict Dozzle to localhost-only binding
Bind Dozzle to 127.0.0.1 to prevent exposure on external/LAN interfaces.
Previously bound to 0.0.0.0, allowing unauthenticated access to container
logs from any network-reachable host.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:02:48 -04:00
Scott Idem
47fe502dc1 Minor clean up 2026-04-19 15:24:28 -04:00
Scott Idem
a56213569a docs: expand .env.default comments for API and DB tuning settings
Updated AE_API_GUNICORN_WORKERS default from 2 → 4 based on stress
testing (nearly 2x throughput improvement confirmed). Added detailed
comments to Gunicorn, DB pool, and connection tuning settings explaining
what each parameter does, how they interact, and capacity planning math.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-17 18:55:28 -04:00
Scott Idem
8d1c27471f feat: expose DB pool_size and max_overflow as env vars
Documents AE_DB_POOL_SIZE and AE_DB_POOL_MAX_OVERFLOW in .env.default
with per-replica connection math comment for capacity planning.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-17 18:06:19 -04:00
Scott Idem
d1ed06a4c4 fix: resolve logrotate permission issues in maintenance container 2026-04-03 17:10:22 -04:00
Scott Idem
3c6b67b149 chore: unify timezone and implement containerized log rotation 2026-04-03 17:06:34 -04:00
Scott Idem
75fc650ba8 docs(cheatsheet): update multi-stack isolation section with full container name var list
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 19:20:52 -04:00
Scott Idem
c136c2e50c chore(env): clean up .env.default and parameterize container names
- Remove 16 dead variables (OSIT_ENV, AE_API_ENV, AE_APP_ENV, AE_FLASK_APP_SRC,
  AE_DB_ROOT_PASSWORD, OSIT_WEB_HTTPS_PORT, 5x DOCKER_AE_*_EXTRA_HOST,
  CONTAINER_AE_API/APP/MARIADB/PMA)
- Add missing vars: AE_NETWORK_NAME, CONTAINER_DOZZLE, AE_DOZZLE_PORT
- Parameterize hardcoded container names in compose: CONTAINER_MARIADB,
  CONTAINER_PMA, CONTAINER_AE_OPS (all with :-default fallbacks)
- Fix AE_DB_EXTERNAL_PORT default: 3306 → 32768 (avoids host MariaDB conflict)
- Reorganize: AE_APP_GATEWAY_PORT moved next to AE_API_GATEWAY_PORT

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 19:02:46 -04:00
Scott Idem
4f15386d93 docs: update CHEATSHEET and README for new build/deploy commands
Replace AE_APP_BUILD_MODE=staging references and old docker compose
build-ui instructions with current build-docker-* and deploy-remote-*
Makefile targets.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 18:05:43 -04:00
Scott Idem
352cca8a27 fix(compose): update BUILD_MODE fallback from staging to dev
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 17:28:10 -04:00
Scott Idem
dbfa9754d9 chore(deploy): add deploy.sh remote script, update Makefile
- deploy.sh: SSH-triggered deploy for prod and test environments on
  srv-nyx (linode.oneskyit.com). Pulls repos, builds ae_app container
  with correct BUILD_MODE, restarts ae_api.
- Makefile: rename build-ui → build-docker-dev/test/prod to match new
  naming convention; add deploy-remote-test and deploy-remote-prod targets
- .env.default: AE_APP_BUILD_MODE staging → dev (from prior session)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 17:18:52 -04:00
Scott Idem
bb437ce5cb chore(env): add .env.default template and track it in .gitignore
The Docker env project had no committed .env template — new contributors
had to reverse-engineer the required variables from the compose files.

Added .env.default with all required variables, secrets replaced with XXXX,
and comments explaining each section. Notable: AE_API_GUNICORN_TIMEOUT is
documented as 900 (needed for long ffmpeg video jobs like clip_video).

Updated .gitignore to whitelist .env.default via !.env.default.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 14:08:11 -04:00
Scott Idem
bd035f8c17 fix(nginx,gunicorn): raise send_timeout and proxy_send_timeout for long-running endpoints
Nginx was closing the client connection after exactly 60 seconds on requests
like clip_video (ffmpeg, 5-40 min) because send_timeout and proxy_send_timeout
both default to 60s. proxy_read_timeout was already 2100s but the other two
timeouts were still at defaults.

With proxy_buffering off, Nginx holds the write path to the client open as soon
as the upstream connection is established. If the upstream sends no data for 60s
(e.g. ffmpeg processing), Nginx treats the idle write path as stalled and closes
the client connection, logging 499 (Client Closed Request).

Fixed: raise proxy_send_timeout and send_timeout to 2100s to match
proxy_read_timeout in the main location block.

Also raised the Gunicorn default timeout from 30s to 120s in gunicorn_conf.py
as a belt-and-suspenders measure (AE_API_GUNICORN_TIMEOUT env var takes precedence).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 14:05:22 -04:00
Scott Idem
6fd6899879 fix(gunicorn): set control_socket to /dev/shm path
Newer gunicorn (post-23.0.0) added _get_control_socket_path() which
calls os.path.isabs() on the value — crashing when it is None.
Point the socket to /dev/shm (already used for worker_tmp_dir) so it
is writable inside the container and satisfies the new gunicorn code.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-24 20:24:12 -04:00
Scott Idem
cd208ef25c Clean up of old stuff 2026-03-24 17:07:19 -04:00
Scott Idem
0d81958bfc Cleaning up old legacy files. Sorry, no more Flask. 2026-03-24 15:51:44 -04:00
Scott Idem
8c9d263afb Disable Gunicorn control_socket in FastAPI configs
Gunicorn 25.1.0+ enables a 'control_socket' by default, which creates
a root-owned 'gunicorn.ctl' file in the chdir directory. When this
directory is a volume mount (as in our dev/test setups), it causes
permission errors during Docker build context gathering.

This change explicitly sets 'control_socket = None' to prevent the
creation of this file.
2026-03-24 15:41:54 -04:00
Scott Idem
facf453991 Added .dockerignore to this file to help with a build issue. 2026-03-24 15:10:29 -04:00
Scott Idem
90a42a68b3 More domains here 2026-03-12 03:15:58 -04:00
Scott Idem
22efb9c832 More sub domains 2026-03-12 03:00:26 -04:00
Scott Idem
1d7200639c fix: moved API healthcheck to docker-compose to override base image defaults and force port 5005. 2026-03-12 02:53:13 -04:00
Scott Idem
f636c021bc docs: updated README and CHEATSHEET with multi-stack and shared network architecture. 2026-03-12 02:28:43 -04:00
Scott Idem
0072a16c25 fix: introduced aether_shared_net to allow isolated stacks to reach shared DB/Redis. 2026-03-12 02:22:35 -04:00
Scott Idem
d80e2aa1ff Adding more server names... 2026-03-12 02:04:42 -04:00
Scott Idem
26e943b066 fix: added AE_NETWORK_NAME variable to isolate networks between stacks. 2026-03-12 01:40:19 -04:00
Scott Idem
ef54720e78 chore: cleaned up Nginx server_name conflicts and fixed API healthcheck port. 2026-03-12 01:22:44 -04:00
Scott Idem
122ae1efc6 fix: added AE_DOZZLE_PORT variable to resolve port conflicts between environments. 2026-03-12 00:40:30 -04:00
Scott Idem
dca75ab990 fix: added CONTAINER_DOZZLE variable to support multiple environments and updated env template. 2026-03-12 00:32:08 -04:00
Scott Idem
7afbc6ffa3 feat: implemented scalable regex for Nginx server_names across App and API. 2026-03-11 23:54:27 -04:00
Scott Idem
4c68cd2ce3 feat: added test-demo and test-api domains to Nginx server_name lists. 2026-03-11 23:50:25 -04:00
Scott Idem
55350ddd7c Add more server names.... 2026-03-11 23:13:45 -04:00
Scott Idem
a7b6112f4d chore: disabled internal SSL/HTTPS in Nginx configs to support host-level SSL termination. 2026-03-11 23:05:51 -04:00
Scott Idem
14173cfc22 feat: Added Makefile for fast stack management and optimized API volume mounts. 2026-03-11 16:15:08 -04:00
Scott Idem
50f4ddf39d chore: Remove mounted API config file — now lives in aether_api_fastapi repo
API config is no longer injected via volume mount. app/config.py in the
aether_api_fastapi repo reads all settings directly from env vars (.env).
Updated README to reflect the new config location.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-10 18:45:18 -04:00
Scott Idem
1aa4186f4a docs: update README and CHEATSHEET to reflect final correct architecture
- Traffic diagram corrected: both app and api route through ae_web_dev
- Removed outdated 'two options' scaling section for ae_app
- Added port reference table to CHEATSHEET
- Scaling section simplified: change replicas, done, home nginx never changes
- localhost link corrected from 3001 to 8888 (LAN HTTP via ae_web_dev)
2026-03-10 18:24:31 -04:00
Scott Idem
57ec65144d refactor: route ae_app through ae_web_dev like ae_api
Both API and App now scale transparently via Docker DNS round-robin.
Home server nginx points at a single port for each service:
  - workstation:5060 -> ae_web_dev:80 -> ae_api replicas
  - workstation:3001 -> ae_web_dev:80 -> ae_app replicas

ae_app no longer needs host port bindings. AE_APP_REPLICAS scales freely.
HTTPS (443) commented out -- SSL terminates at home server, not internally.
2026-03-10 18:16:06 -04:00
Scott Idem
0ea5373390 docs: clarify traffic architecture and ae_app vs ae_api scaling options 2026-03-10 17:50:55 -04:00
Scott Idem
16f98bc93d fix: use port range for ae_app to support scaling with host port binding
Replace single AE_APP_NODE_PORT with AE_APP_NODE_PORT_RANGE (e.g. 3001-3006).
Docker assigns one port from the range to each replica, enabling external
nginx upstreams to reference individual ports per instance.

ae_api needs no host ports -- it scales via Docker DNS through ae_web_dev.
2026-03-10 17:40:26 -04:00
Scott Idem
055afda9c4 fix: restore ae_app host port mapping, scale=1 required for static port bind 2026-03-10 17:30:32 -04:00
Scott Idem
6d7cd97bd5 fix: remove host port mapping from ae_app service
With scale > 1, multiple replicas cannot bind to the same host port.
ae_app does not need a host port -- nginx routes to it internally via
the Docker service name 'ae_app:3000' with round-robin load balancing.
Removed ports: '${AE_APP_NODE_PORT}:3000' to fix the port conflict.
2026-03-10 17:11:38 -04:00
Scott Idem
83770ffc00 chore: add SvelteKit nginx config, clean up Flask upstream, wire AE_APP_REPLICAS
- Add conf/nginx/site-enabled_aether_app_svelte_node.conf (SvelteKit node upstream)
- Clean up site-enabled_aether_flask_gunicorn.conf: remove dead comments, update
  upstream to app-node:3000
- docker-compose.yml: ae_app scale now uses ${AE_APP_REPLICAS:-1} instead of hardcoded 1
2026-03-10 16:11:46 -04:00
Scott Idem
f51f12755c docs: Simplified setup instructions by removing redundant symlink steps. 2026-03-10 15:29:54 -04:00
Scott Idem
5fd57dc11a chore: Updated ae_api orchestration to use project-internal build context and Dockerfile. 2026-03-10 15:23:13 -04:00
Scott Idem
decba7f7c8 chore: Pruned redundant FastAPI dependencies (watchgod, six, python-dotenv, itsdangerous, baize, async-timeout). 2026-03-10 15:01:38 -04:00
Scott Idem
8abc3b4c7e docs: Update for unified SvelteKit + FastAPI architecture. Added autonomous SvelteKit build process and updated cheatsheet commands. 2026-03-10 13:33:23 -04:00
Scott Idem
b4866c2f23 docs: modernize README and finalize environment synchronization
Updated README.md to reflect V3 architecture, documented the physical database management suite, and finalized synchronization between env.default and active .env configuration.
2026-02-06 13:35:20 -05:00
Scott Idem
49539d52c1 chore(env): synchronize env.default and stabilize API config
Updated env.default with self-documenting comments and all active environment variables. Hardened conf/aether_api_config.py to ensure SMTP and FILES_PATH dictionaries are preserved during refactors. Integrated v3 websocket routes into Nginx template.
2026-02-06 13:15:48 -05:00
Scott Idem
a303b23d54 More comments 2026-01-13 13:59:12 -05:00
Scott Idem
6297df094d Bug fix for the restore script. It now deletes the temp backup directories automatically. 2026-01-13 12:47:08 -05:00
Scott Idem
ea45d99f13 Enhance DB automation: Added conference export, absolute path restore logic, and automated multi-user credential resetting. Updated README and CHEATSHEET. 2026-01-12 20:28:02 -05:00