Docs: Consolidate V3 standards and cleanup documentation directory

documentation/V3_DEVELOPMENT_STANDARDS.md

@@ -0,0 +1,47 @@
+# Aether V3 Development Standards & Strategy
+
+This document serves as the master guide for the Aether API V3. It combines core architectural principles with the finalized infrastructure standards established in January 2026.
+
+## 1. Core Principles
+
+-   **`id_random` Primary**: `id_random` (URL-safe string) is the only identifier exposed to clients. Internal integer `id` is a private implementation detail.
+-   **Singular naming**: All object types and prefixes use singular form (e.g., `journal`, not `journals`).
+-   **Inheritance**: All models must inherit from `CoreObject` to ensure consistent base fields.
+-   **Separation of Concerns**: Business logic lives in `app/methods/`, CRUD helpers in `app/lib_sql_crud.py`, and routing in `app/routers/`.
+
+## 2. Infrastructure Standards
+
+Finalized Jan 15, 2026, to ensure boot stability.
+
+### Application Entry Point (`app/main.py`)
+-   **Registry Pattern**: All routers are registered via `app/routers/registry.py`.
+-   **Lifespan Management**: All startup tasks (logging setup, DB bootstrap, engine refresh) MUST reside within the `@contextlib.asynccontextmanager` lifespan.
+-   **No Top-Level Logic**: No SQL queries or heavy initialization should execute at the module level.
+
+### Database Layering
+-   **`lib_sql_core.py`**: The foundational "Source of Truth" for the SQLAlchemy engine and global `db` connection.
+-   **`lib_sql_crud.py`**: High-level logic for `sql_insert`, `sql_select`, etc.
+-   **`db_sql.py`**: A backward-compatible facade. code should import from here.
+
+### Logging
+-   **Explicit Setup**: `setup_logging(settings)` must be called during the lifespan startup.
+-   **`lib_log_v3.py`**: New home for logging configuration and the `logger_reset` decorator.
+
+## 3. V3 CRUD Strategy
+
+### URL Structure
+-   **Top-Level**: `/v3/crud/{obj_type}/`
+-   **Nested**: `/v3/crud/{parent_type}/{parent_id_random}/{obj_type}/` (Enforces parent ownership).
+
+### Search API
+-   **POST Based**: Complex filtering is handled via `POST /search` with a JSON body containing `and`, `or`, and `not` logic.
+-   **Hybrid Filtering**: (Proposed) Query parameters should append simple standard filters (e.g., `?enabled=true`) to the complex body logic.
+
+### Response Views (Proposed)
+-   Implement a `view` parameter (e.g., `?view=rich`) to allow clients to request joined data without using legacy `use_alt_tbl` flags.
+
+## 4. Stability Rules
+
+1.  **Baby Step Testing**: Restart Docker and verify root health after *every* modular change.
+2.  **Avoid Shadowing**: Never name a module part of the `app.` package the same as a common instance variable (e.g., avoid `app.middleware` package if you use `app = FastAPI()`).
+3.  **Deferred Imports**: Use `from app.db_sql import ...` *inside* functions in library modules to prevent circular dependency traps.

documentation/docker-compose_2025-12-31_working.yml

@@ -1,372 +0,0 @@
-services:
-    web:
-        restart: no # unless-stopped
-        container_name: ${CONTAINER_WEB}
-        build:
-            # context: ./builds
-            context: ./
-            dockerfile: aether_nginx.Dockerfile
-        env_file:
-            - ./.env
-        environment:
-            - PUID=1000
-            - PGID=1000
-
-            # NOTE: This does not seem to work with nginx yet???
-            # NOTE: Use the template directory and .template extension for the .conf files.
-            # - NGINX_SERVER_NAMES="dev-demo.oneskyit.com dev-example.oneskyit.com"
-            # - NGINX_SERVER_NAMES="dev-idaa.oneskyit.com dev-ishlt.oneskyit.com"
-            # - AE_DOMAIN_LIST:'dev-aapor.oneskyit.com dev-businessgroup.oneskyt.com dev-cmsc.oneskyit.com dev-idaa.oneskyit.com dev-ishlt.oneskyit.com dev-ncsd.oneskyit.com dev-npa.oneskyit.com dev-rli.oneskyit.com'
-            # - NGINX_SERVER_NAMES="flask_gunicorn.localhost demo.localhost dev.localhost dev.oneskyit.com dev-app.oneskyit.com dev-connect.oneskyit.com *.dev-connect.oneskyit.com dev-demo.oneskyit.com *.dev-demo.oneskyit.com dev-aapor.oneskyit.com *.dev-aapor.oneskyit.com dev-businessgroup.oneskyt.com *.dev-businessgroup.oneskyt.com dev-cmsc.oneskyit.com *.dev-cmsc.oneskyit.com dev-idaa.oneskyit.com *.dev-idaa.oneskyit.com dev-ishlt.oneskyit.com *.dev-ishlt.oneskyit.com dev-ncsd.oneskyit.com *.dev-ncsd.oneskyit.com dev-npa.oneskyit.com *.dev-npa.oneskyit.com dev-rli.oneskyit.com *.dev-rli.oneskyit.com test-app.oneskyit.com"
-            - NGINX_SERVER_NAMES="flask_gunicorn.localhost demo.localhost dev.localhost dev.oneskyit.com dev-app.oneskyit.com dev-connect.oneskyit.com dev-demo.oneskyit.com dev-aacc.oneskyit.com dev-aapor.oneskyit.com dev-ascm.oneskyit.com dev-businessgroup.oneskyt.com dev-chow.oneskyit.com dev-cmsc.oneskyit.com dev-idaa.oneskyit.com dev-ishlt.oneskyit.com dev-lci.oneskyit.com dev-ncsd.oneskyit.com dev-npa.oneskyit.com dev-rli.oneskyit.com test-app.oneskyit.com"
-        ports:
-            - "${OSIT_WEB_HTTP_PORT}:80"
-            - "${OSIT_WEB_HTTPS_PORT}:443"
-            # - "80:80"
-            # - "443:443"
-            # - "8181:80"
-            # - "8443:443"
-        # networks:
-        #   - local-net
-        volumes:
-            - ./srv/html_php:/srv/html_php
-            - ./srv/oneskyit_site:/srv/oneskyit_site
-
-            - ./srv/hosted_files_ln:/srv/hosted_files
-            - ./srv/hosted_tmp_ln:/srv/hosted_tmp
-
-            # - ./conf/nginx/nginx.conf:/etc/nginx/nginx.conf
-            - ./conf/nginx/options-ssl-nginx.conf:/etc/nginx/options-ssl-nginx.conf
-            # - ./conf/nginx/other.conf:/etc/nginx/conf.d/other.conf
-
-            - ./conf/nginx/site.conf:/etc/nginx/conf.d/0_site.conf
-
-            - ./conf/nginx/site-enabled_aether_fastapi_gunicorn.conf:/etc/nginx/templates/site-enabled_aether_fastapi_gunicorn.conf.template
-            # - ./conf/nginx/site-enabled_aether_api_v5_fastapi_gunicorn.conf:/etc/nginx/templates/site-enabled_aether_api_v5_fastapi_gunicorn.conf.template
-            - ./conf/nginx/site-enabled_aether_flask_gunicorn.conf:/etc/nginx/templates/site-enabled_aether_flask_gunicorn.conf.template
-
-            - ./conf/certs/oneskyit_wild_fullchain.pem:/etc/certs/fullchain_wild.pem
-            - ./conf/certs/oneskyit_wild_privkey.pem:/etc/certs/privkey_wild.pem
-            - ./conf/certs/oneskyit.com_fullchain.pem:/etc/certs/fullchain.pem
-            - ./conf/certs/oneskyit.com_privkey.pem:/etc/certs/privkey.pem
-            - ./conf/certs/ssl-dhparams.pem:/etc/certs/ssl-dhparams.pem
-
-            - ./logs/web:/logs
-        depends_on:
-            # - php7
-            # - aether_api_gunicorn
-            - aether_api_gunicorn_red
-            - aether_api_gunicorn_green
-            # - aether_api_v5_gunicorn
-            - aether_app_gunicorn
-
-
-    # Need to fix the memory overcommit warning from Redis
-    # https://ourcodeworld.com/articles/read/2083/how-to-remove-redis-warning-on-docker-memory-overcommit-must-be-enabled
-    redis:
-        restart: always
-        container_name: ${CONTAINER_REDIS}
-        image: redis
-        # By default redis saves every 3600 seconds if there is at least 1 change.
-        command: redis-server --save "" --loglevel warning
-        # command: redis-server --save 60 1 --loglevel warning
-        # build:
-        # ports:
-        #     # host to image
-        #     # default port is 6379
-        #     - "${AE_REDIS_PORT}:6379"
-
-    # API - Default
-    # aether_api_gunicorn:
-    #     restart: always
-    #     container_name: ${CONTAINER_AE_API}
-    #     build:
-    #         # context: ./builds
-    #         context: ./
-    #         dockerfile: aether_fastapi_gunicorn.Dockerfile
-    #     env_file:
-    #         - ./.env
-    #     environment:
-    #         - PUID=1000
-    #         - PGID=1000
-    #     ports:
-    #         - "${AE_API_GUNICORN_PORT}:5005"
-    #     extra_hosts:
-    #         - "${DOCKER_AE_SERVER_EXTRA_HOST}"
-    #         - "${DOCKER_AE_APP_SERVER_EXTRA_HOST}"
-    #         - "${DOCKER_AE_API_SERVER_EXTRA_HOST}"
-    #         - "${DOCKER_AE_API_BAK_SERVER_EXTRA_HOST}"
-    #         - "${DOCKER_AE_DB_SERVER_EXTRA_HOST}"
-    #         - "linode.oneskyit.com:104.237.143.4"
-    #         # - "db.oneskyit.com:104.237.143.4"
-    #     # expose:
-    #     #   - 5005
-    #     # networks:
-    #     #   - local-net
-    #     volumes:
-    #         - ./conf/aether_fastapi_gunicorn_conf.py:/conf/gunicorn_fastapi_conf.py
-    #         - ./conf/aether_fastapi_requirements_current.txt:/requirements_current.txt
-    #         - ./conf/aether_api_config.py:/srv/aether_api/app/config.py
-
-    #         - ./logs/ae_api:/logs
-
-    #         - ./srv/aether_api_ln:/srv/aether_api
-    #         - ./srv/hosted_files_ln:/srv/hosted_files
-    #         - ./srv/hosted_tmp_ln:/srv/hosted_tmp
-
-    #         # - ./tmp/ae_api:/tmp
-    #         # - ./tmp/root/aether_fastapi_requirements_current.txt:/aether_fastapi_requirements_current.txt
-    #         # - ./temp/ae_api/aether_fastapi_requirements_current.txt:/temp/aether_fastapi_requirements_current.txt
-    #         # - ./tmp/test:/var
-    #     # links:
-    #     #     - redis
-    #     depends_on:
-    #         - redis
-    #     stdin_open: true # docker run -i
-    #     tty: true        # docker run -t
-
-    # API - Red
-    aether_api_gunicorn_red:
-        restart: always
-        container_name: ${CONTAINER_AE_API_RED}
-        build:
-            # context: ./builds
-            context: ./
-            dockerfile: aether_fastapi_gunicorn.Dockerfile
-        env_file:
-            - ./.env
-        ports:
-            - "${AE_API_GUNICORN_PORT_RED}:5005"
-        extra_hosts:
-            - "${DOCKER_AE_SERVER_EXTRA_HOST}"
-            - "${DOCKER_AE_APP_SERVER_EXTRA_HOST}"
-            - "${DOCKER_AE_API_SERVER_EXTRA_HOST}"
-            - "${DOCKER_AE_API_BAK_SERVER_EXTRA_HOST}"
-            - "${DOCKER_AE_DB_SERVER_EXTRA_HOST}"
-            - "linode.oneskyit.com:104.237.143.4"
-            # - "db.oneskyit.com:104.237.143.4"
-        # expose:
-        #   - 5005
-        # networks:
-        #   - local-net
-        volumes:
-            - ./conf/aether_fastapi_gunicorn_conf.py:/conf/gunicorn_fastapi_conf.py
-            - ./conf/aether_fastapi_requirements_current.txt:/requirements_current.txt
-            - ./conf/aether_api_config.py:/srv/aether_api/app/config.py
-
-            - ./logs/ae_api:/logs
-
-            - ./srv/aether_api_ln:/srv/aether_api
-            - ./srv/hosted_files_ln:/srv/hosted_files
-            - ./srv/hosted_tmp_ln:/srv/hosted_tmp
-
-            # - ./tmp/ae_api:/tmp
-            - ./temp/ae_api:/temp
-        # links:
-        #     - redis
-        depends_on:
-            - redis
-        stdin_open: true # docker run -i
-        tty: true        # docker run -t
-
-    # API - Green
-    aether_api_gunicorn_green:
-        restart: always
-        container_name: ${CONTAINER_AE_API_GREEN}
-        build:
-            # context: ./builds
-            context: ./
-            dockerfile: aether_fastapi_gunicorn.Dockerfile
-        env_file:
-            - ./.env
-        ports:
-            - "${AE_API_GUNICORN_PORT_GREEN}:5005"
-        extra_hosts:
-            - "${DOCKER_AE_SERVER_EXTRA_HOST}"
-            - "${DOCKER_AE_APP_SERVER_EXTRA_HOST}"
-            - "${DOCKER_AE_API_SERVER_EXTRA_HOST}"
-            - "${DOCKER_AE_API_BAK_SERVER_EXTRA_HOST}"
-            - "${DOCKER_AE_DB_SERVER_EXTRA_HOST}"
-            - "linode.oneskyit.com:104.237.143.4"
-            # - "db.oneskyit.com:104.237.143.4"
-        # expose:
-        #   - 5005
-        # networks:
-        #   - local-net
-        volumes:
-            - ./conf/aether_fastapi_gunicorn_conf.py:/conf/gunicorn_fastapi_conf.py
-            - ./conf/aether_fastapi_requirements_current.txt:/requirements_current.txt
-            - ./conf/aether_api_config.py:/srv/aether_api/app/config.py
-
-            - ./logs/ae_api:/logs
-
-            - ./srv/aether_api_ln:/srv/aether_api
-            - ./srv/hosted_files_ln:/srv/hosted_files
-            - ./srv/hosted_tmp_ln:/srv/hosted_tmp
-
-            # - ./tmp/ae_api:/tmp
-            - ./temp/ae_api:/temp
-        # links:
-        #     - redis
-        depends_on:
-            - redis
-        stdin_open: true # docker run -i
-        tty: true        # docker run -t
-
-    # API - Blue
-    # aether_api_gunicorn_blue:
-    #     restart: always
-    #     container_name: ${CONTAINER_AE_API_BLUE}
-    #     build:
-    #         # context: ./builds
-    #         context: ./
-    #         dockerfile: aether_fastapi_gunicorn.Dockerfile
-    #     env_file:
-    #         - ./.env
-    #     ports:
-    #         - "${AE_API_GUNICORN_PORT_BLUE}:5005"
-    #     extra_hosts:
-    #         - "${DOCKER_AE_SERVER_EXTRA_HOST}"
-    #         - "${DOCKER_AE_APP_SERVER_EXTRA_HOST}"
-    #         - "${DOCKER_AE_API_SERVER_EXTRA_HOST}"
-    #         - "${DOCKER_AE_API_BAK_SERVER_EXTRA_HOST}"
-    #         - "${DOCKER_AE_DB_SERVER_EXTRA_HOST}"
-    #         - "linode.oneskyit.com:104.237.143.4"
-    #         # - "db.oneskyit.com:104.237.143.4"
-    #     # expose:
-    #     #   - 5005
-    #     # networks:
-    #     #   - local-net
-    #     volumes:
-    #         - ./conf/aether_fastapi_gunicorn_conf.py:/conf/gunicorn_fastapi_conf.py
-    #         - ./conf/aether_fastapi_requirements_current.txt:/requirements_current.txt
-    #         - ./conf/aether_api_config.py:/srv/aether_api/app/config.py
-
-    #         - ./logs/ae_api:/logs
-
-    #         - ./srv/aether_api_ln:/srv/aether_api
-    #         - ./srv/hosted_files_ln:/srv/hosted_files
-    #         - ./srv/hosted_tmp_ln:/srv/hosted_tmp
-
-    #         # - ./tmp/ae_api:/tmp
-    #         - ./temp/ae_api:/temp
-    #     # links:
-    #     #     - redis
-    #     depends_on:
-    #         - redis
-    #     stdin_open: true # docker run -i
-    #     tty: true        # docker run -t
-
-    # API - Black
-    # aether_api_gunicorn_black:
-    #     restart: always
-    #     container_name: ${CONTAINER_AE_API_BLACK}
-    #     build:
-    #         # context: ./builds
-    #         context: ./
-    #         dockerfile: aether_fastapi_gunicorn.Dockerfile
-    #     env_file:
-    #         - ./.env
-    #     ports:
-    #         - "${AE_API_GUNICORN_PORT_BLACK}:5005"
-    #     extra_hosts:
-    #         - "${DOCKER_AE_SERVER_EXTRA_HOST}"
-    #         - "${DOCKER_AE_APP_SERVER_EXTRA_HOST}"
-    #         - "${DOCKER_AE_API_SERVER_EXTRA_HOST}"
-    #         - "${DOCKER_AE_API_BAK_SERVER_EXTRA_HOST}"
-    #         - "${DOCKER_AE_DB_SERVER_EXTRA_HOST}"
-    #         - "linode.oneskyit.com:104.237.143.4"
-    #         # - "db.oneskyit.com:104.237.143.4"
-    #     volumes:
-    #         - ./conf/aether_fastapi_gunicorn_conf.py:/conf/gunicorn_fastapi_conf.py
-    #         - ./conf/aether_fastapi_requirements_current.txt:/requirements_current.txt
-    #         - ./conf/aether_api_config.py:/srv/aether_api/app/config.py
-
-    #         - ./logs/ae_api:/logs
-
-    #         - ./srv/aether_api_ln:/srv/aether_api
-    #         - ./srv/hosted_files_ln:/srv/hosted_files
-    #         - ./srv/hosted_tmp_ln:/srv/hosted_tmp
-
-    #         - ./temp/ae_api:/temp
-    #     depends_on:
-    #         - redis
-    #     stdin_open: true
-    #     tty: true
-
-    # API - White
-    # aether_api_gunicorn_white:
-    #     restart: always
-    #     container_name: ${CONTAINER_AE_API_WHITE}
-    #     build:
-    #         context: ./
-    #         dockerfile: aether_fastapi_gunicorn.Dockerfile
-    #     env_file:
-    #         - ./.env
-    #     ports:
-    #         - "${AE_API_GUNICORN_PORT_WHITE}:5005"
-    #     extra_hosts:
-    #         - "${DOCKER_AE_SERVER_EXTRA_HOST}"
-    #         - "${DOCKER_AE_APP_SERVER_EXTRA_HOST}"
-    #         - "${DOCKER_AE_API_SERVER_EXTRA_HOST}"
-    #         - "${DOCKER_AE_API_BAK_SERVER_EXTRA_HOST}"
-    #     volumes:
-    #         - ./conf/aether_fastapi_gunicorn_conf.py:/conf/gunicorn_fastapi_conf.py
-    #         - ./conf/aether_fastapi_requirements_current.txt:/requirements_current.txt
-    #         - ./conf/aether_api_config.py:/srv/aether_api/app/config.py
-
-    #         - ./logs/ae_api:/logs
-
-    #         - ./srv/aether_api_ln:/srv/aether_api
-    #         - ./srv/hosted_files_ln:/srv/hosted_files
-    #         - ./srv/hosted_tmp_ln:/srv/hosted_tmp
-
-    #         - ./temp/ae_api:/temp
-    #     depends_on:
-    #         - redis
-    #     stdin_open: true
-    #     tty: true
-
-
-    aether_app_gunicorn:
-        restart: always
-        container_name: ${CONTAINER_AE_APP}
-        build:
-            # context: ./builds
-            context: ./
-            dockerfile: aether_flask_gunicorn.Dockerfile
-        # image: tiangolo/uvicorn-gunicorn:latest
-        env_file:
-            - ./.env
-        ports:
-            - "${AE_APP_GUNICORN_PORT}:5005"
-        # expose:
-        #   - 5005
-        # networks:
-        #   - local-net
-        extra_hosts:
-             - "${DOCKER_AE_SERVER_EXTRA_HOST}"
-             - "${DOCKER_AE_API_SERVER_EXTRA_HOST}"
-             - "${DOCKER_AE_API_BAK_SERVER_EXTRA_HOST}"
-             # - "${DOCKER_AE_API_V5_SERVER_EXTRA_HOST}"
-             # - dev-api.oneskyit.com:192.168.32.20
-        volumes:
-            - ./conf/aether_flask_gunicorn_conf.py:/conf/gunicorn_flask_conf.py
-            - ./conf/aether_flask_requirements_current.txt:/requirements_current.txt
-            - ./conf/aether_app_config.py:/srv/aether_app/flask_config_v2.py
-            - ./logs/ae_app:/logs
-            # - ./logs/aether_flask_gunicorn_access.log:/logs/gunicorn_access.log
-            # - ./logs/aether_flask_gunicorn_error.log:/logs/gunicorn_error.log
-            # - ./logs/aether_app.log:/logs/aether_app.log
-            # - ./logs/aether_app_warning.log:/logs/aether_app_warning.log
-            - ./srv/aether_app_ln:/srv/aether_app
-            - ./srv/hosted_files_ln:/srv/hosted_files
-            - ./srv/hosted_tmp_ln:/srv/hosted_tmp
-
-            - ./tmp/ae_app:/tmp
-        depends_on:
-            # - aether_api_gunicorn
-            - aether_api_gunicorn_red
-            - aether_api_gunicorn_green
-        stdin_open: true # docker run -i
-        tty: true        # docker run -t