diff --git a/README.md b/README.md index 92d53cc..03bd95b 100644 --- a/README.md +++ b/README.md @@ -214,12 +214,12 @@ To configure the mailman-web container to send emails, add this to your ``` EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend' -EMAIL_HOST = '172.19.199.1' +EMAIL_HOST = 'smtp.example.com' EMAIL_PORT = 25 ``` Alternatively, you can use the environment variables `SMTP_HOST` (defaults to -`172.19.199.1`), `SMTP_PORT` (defaults to `25`), `SMTP_HOST_USER` (defaults to +the container's gateway), `SMTP_PORT` (defaults to `25`), `SMTP_HOST_USER` (defaults to an empty string), `SMTP_HOST_PASSWORD` (defaults to an empty string) and `SMTP_USE_TLS` (defaults to `False`). @@ -288,7 +288,7 @@ See [the nginx configuration][17] as an example. This command will do several things, most importantly: - Run a wsgi server using [`uwsgi`][7] for the Mailman's Django-based web - frontend listening on http://172.19.199.3:8000/. It will run 2 worker + frontend listening on port 8000. It will run 2 worker processes with 4 threads each. You may want to change the setting `ALLOWED_HOSTS` in the settings before deploying the application in production. @@ -297,9 +297,9 @@ This command will do several things, most importantly: mentioned in the `docker-compose.yaml`. You will have to change configuration files too if you change any of these. -- Run mailman-core listening an LMTP server at http://172.19.199.2:8024/ for - messages from your MTA. You will have to configure your MTA to send messages at - this address. +- Run mailman-core listening on port 8001 for REST API and port 8024 (LMTP + server) for messages from your MTA. You will have to configure your MTA to + send messages at this address. Some more details about what the above system achieves is mentioned below. If you are only going to deploy a simple configuration, you don't need to read @@ -307,13 +307,12 @@ this. However, these are very easy to understand if you know how docker works. - First create a bridge network called `mailman` in the `docker-compose.yaml`. It will probably be named something else in your - machine, but it will use the `172.19.199.0/24` as subnet. All the containers + machine. All the containers mentioned (mailman-core, mailman-web, database) will join this network and are - assigned static IPs. The host operating system is available at `172.19.199.1` + assigned static IPs. The host operating system is the default gateway from within these containers. -- Spin off a mailman-core container which has a static IP address of - `172.19.199.2` in the mailman bridge network created above. It has +- Spin off a mailman-core container attached to the mailman bridge network created above. It has GNU Mailman 3 core running inside it. Mailman core's REST API is available at port 8001 and LMTP server listens at port 8024. @@ -354,8 +353,8 @@ The provided docker containers do not have an MTA in-built. You can either run your own MTA inside a container and have them relay emails to the mailman-core container or just install an MTA on the host and have them relay emails. -To use [Exim4][8], it should be setup to relay emails from `172.19.199.3` and -`172.19.199.2`. The mailman specific configuration is provided in the +To use [Exim4][8], it should be setup to relay emails from mailman-core and +mailman-web. The mailman specific configuration is provided in the repository at `core/assets/exim`. There are three files - [25_mm_macros](core/assets/exim/25_mm3_macros) to be placed at @@ -387,7 +386,7 @@ configuration: python:mailman.config.exim4 To use [Postfix][12], edit the `main.cf` configuration file, which is typically at `/etc/postfix/main.cf` on Debian-based operating systems. Add -`172.19.199.2` and `172.19.199.3` to `mynetworks` so it will relay emails from +mailman-core and mailman-web to `mynetworks` so it will relay emails from the containers and add the following configuration lines: ``` @@ -415,9 +414,11 @@ at `/opt/mailman/core/mailman-extra.cfg`. [mta] incoming: mailman.mta.postfix.LMTP outgoing: mailman.mta.deliver.deliver -lmtp_host: 172.19.199.2 +# mailman-core hostname or IP from the Postfix server +lmtp_host: localhost lmtp_port: 8024 -smtp_host: 172.19.199.1 +# Postfix server's hostname or IP from mailman-core +smtp_host: smtp.example.com smtp_port: 25 configuration: /etc/postfix-mailman.cfg ``` @@ -472,7 +473,7 @@ It is advisable to run your Django (interfaced through WSGI server) through an _actual_ webserver in production for better performance. If you are using v0.1.0, the uwsgi server is configured to listen to requests at -`172.19.199.3:8000` using the `HTTP` protocol. Make sure that you preserve the `HOST` +port `8000` using the `HTTP` protocol. Make sure that you preserve the `HOST` header when you proxy the requests from your Web Server. In Nginx you can do that by adding the following to your configuration: @@ -485,7 +486,7 @@ that by adding the following to your configuration: location / { - proxy_pass http://172.19.199.3:8000; + proxy_pass http://localhost:8000; include uwsgi_params; uwsgi_read_timeout 300; proxy_set_header Host $host; @@ -500,7 +501,7 @@ uwsgi ----- Starting from v0.1.1, the uwsgi server is configured to listen to requests at -`172.19.199.3:8000` with the http protocol and `172.19.199.3:8080` for the uwsgi +port `8000` with the http protocol and port `8080` for the uwsgi protocol. **Please make sure that you are using port 8080 for uwsgi protocol.** @@ -519,7 +520,7 @@ To move to uwsgi protocol in the above nginx configuration use this } location / { - uwsgi_pass 172.19.199.3:8080; + uwsgi_pass localhost:8080; include uwsgi_params; uwsgi_read_timeout 300; } diff --git a/core/README.md b/core/README.md index 9c54ab4..2a8c451 100644 --- a/core/README.md +++ b/core/README.md @@ -40,8 +40,8 @@ These are the variables that you MUST change before deploying: These are the variables that you don't need to change if you are using a standard version of docker-compose.yaml from this repository. -- `MM_HOSTNAME`: Which IP should Core bind to for REST API and LMTP. If not - defined output for `hostname -i` command is used. +- `MM_HOSTNAME`: Which hostname or IP should Core bind to for REST API and + LMTP. If not defined output from `hostname` command is used. - `MAILMAN_REST_PORT`: Which port should Core use for the REST API. If not defined the default is `8001`. @@ -55,7 +55,8 @@ standard version of docker-compose.yaml from this repository. - `MTA`: Mail Transfer Agent to use. Either `exim` or `postfix`. Default value is `exim`. - `SMTP_HOST`: IP Address/hostname from which you will be sending - emails. Default value is `172.19.199.1`, which is the address of the Host OS. + emails. Default value is the container's gateway retrieved from: + /sbin/ip route | awk '/default/ { print $3 }' - `SMTP_PORT`: Port used for SMTP. Default is `25`. diff --git a/core/assets/exim/25_mm3_macros b/core/assets/exim/25_mm3_macros index b648263..b823fa2 100644 --- a/core/assets/exim/25_mm3_macros +++ b/core/assets/exim/25_mm3_macros @@ -2,7 +2,9 @@ # /etc/exim4/conf.d/main/25_mm3_macros domainlist mm3_domains=MY_DOMAIN_NAME -MM3_LMTP_HOST=172.19.199.2 +# Depending on your network configuration +#MM3_LMTP_HOST=mailman-core +MM3_LMTP_HOST=localhost MM3_LMTP_PORT=8024 MM3_HOME=/opt/mailman/core/var diff --git a/core/docker-entrypoint.sh b/core/docker-entrypoint.sh index 9a4d0d7..e2e5d56 100755 --- a/core/docker-entrypoint.sh +++ b/core/docker-entrypoint.sh @@ -27,14 +27,16 @@ function wait_for_mysql () { echo "# This file is autogenerated at container startup." > /etc/mailman.cfg # Check if $MM_HOSTNAME is set, if not, set it to the value returned by -# `hostname -i` command to set it to whatever IP address is assigned to the +# `hostname` command to set it to whatever hostname is assigned to the # container. if [[ ! -v MM_HOSTNAME ]]; then - export MM_HOSTNAME=`hostname -i` + export MM_HOSTNAME=`hostname` fi +# SMTP_HOST defaults to the gateway if [[ ! -v SMTP_HOST ]]; then - export SMTP_HOST='172.19.199.1' + export SMTP_HOST=$(/sbin/ip route | awk '/default/ { print $3 }') + echo "SMPT_HOST not specified, using the gateway ($SMTP_HOST) as default" fi if [[ ! -v SMTP_PORT ]]; then diff --git a/docker-compose-mysql.yaml b/docker-compose-mysql.yaml index 271ed9b..4760bb4 100644 --- a/docker-compose-mysql.yaml +++ b/docker-compose-mysql.yaml @@ -16,9 +16,11 @@ services: - DATABASE_TYPE=mysql - DATABASE_CLASS=mailman.database.mysql.MySQLDatabase - HYPERKITTY_API_KEY=someapikey + ports: + - "127.0.0.1:8001:8001" # API + - "127.0.0.1:8024:8024" # LMTP - incoming emails networks: mailman: - ipv4_address: 172.19.199.2 mailman-web: image: maxking/mailman-web:0.3 @@ -37,9 +39,11 @@ services: - HYPERKITTY_API_KEY=someapikey - SECRET_KEY=thisisaverysecretkey - DYLD_LIBRARY_PATH=/usr/local/mysql/lib/ + ports: + - "127.0.0.1:8000:8000" # HTTP + - "127.0.0.1:8080:8080" # uwsgi networks: mailman: - ipv4_address: 172.19.199.3 database: environment: @@ -54,13 +58,6 @@ services: - /opt/mailman/database:/var/lib/mysql networks: mailman: - ipv4_address: 172.19.199.4 networks: mailman: - driver: bridge - ipam: - driver: default - config: - - - subnet: 172.19.199.0/24 diff --git a/docker-compose-postorius.yaml b/docker-compose-postorius.yaml index c9828ff..8d3ab0b 100644 --- a/docker-compose-postorius.yaml +++ b/docker-compose-postorius.yaml @@ -16,9 +16,11 @@ services: - DATABASE_URL=postgres://mailman:mailmanpass@database/mailmandb - DATABASE_TYPE=postgres - DATABASE_CLASS=mailman.database.postgresql.PostgreSQLDatabase + ports: + - "127.0.0.1:8001:8001" # API + - "127.0.0.1:8024:8024" # LMTP - incoming emails networks: mailman: - ipv4_address: 172.19.199.2 mailman-web: image: maxking/postorius:0.3 @@ -36,9 +38,11 @@ services: - DATABASE_URL=postgres://mailman:mailmanpass@database/mailmandb - SECRET_KEY=ksjdbaksdba - UWSGI_STATIC_MAP=/static=/opt/mailman-web-data/static + ports: + - "127.0.0.1:8000:8000" # HTTP + - "127.0.0.1:8080:8080" # uwsgi networks: mailman: - ipv4_address: 172.19.199.3 database: environment: @@ -51,13 +55,6 @@ services: - /opt/mailman/database:/var/lib/postgresql/data networks: mailman: - ipv4_address: 172.19.199.4 networks: mailman: - driver: bridge - ipam: - driver: default - config: - - - subnet: 172.19.199.0/24 diff --git a/docker-compose.yaml b/docker-compose.yaml index 153d0ba..15b74f7 100644 --- a/docker-compose.yaml +++ b/docker-compose.yaml @@ -17,9 +17,11 @@ services: - DATABASE_TYPE=postgres - DATABASE_CLASS=mailman.database.postgresql.PostgreSQLDatabase - HYPERKITTY_API_KEY=someapikey + ports: + - "127.0.0.1:8001:8001" # API + - "127.0.0.1:8024:8024" # LMTP - incoming emails networks: mailman: - ipv4_address: 172.19.199.2 mailman-web: image: maxking/mailman-web:0.3 @@ -36,9 +38,11 @@ services: - DATABASE_TYPE=postgres - DATABASE_URL=postgres://mailman:mailmanpass@database/mailmandb - HYPERKITTY_API_KEY=someapikey + ports: + - "127.0.0.1:8000:8000" # HTTP + - "127.0.0.1:8080:8080" # uwsgi networks: mailman: - ipv4_address: 172.19.199.3 database: environment: @@ -50,13 +54,6 @@ services: - /opt/mailman/database:/var/lib/postgresql/data networks: mailman: - ipv4_address: 172.19.199.4 networks: mailman: - driver: bridge - ipam: - driver: default - config: - - - subnet: 172.19.199.0/24 diff --git a/postorius/docker-entrypoint.sh b/postorius/docker-entrypoint.sh index 4af5773..26c7e38 100755 --- a/postorius/docker-entrypoint.sh +++ b/postorius/docker-entrypoint.sh @@ -53,6 +53,11 @@ function check_or_create () { # END # } +# SMTP_HOST defaults to the gateway +if [[ ! -v SMTP_HOST ]]; then + export SMTP_HOST=$(/sbin/ip route | awk '/default/ { print $3 }') +fi + # Check if $SECRET_KEY is defined, if not, bail out. if [[ ! -v SECRET_KEY ]]; then echo "SECRET_KEY is not defined. Aborting." diff --git a/postorius/mailman-web/settings.py b/postorius/mailman-web/settings.py index 3d9f815..79cc70c 100644 --- a/postorius/mailman-web/settings.py +++ b/postorius/mailman-web/settings.py @@ -29,6 +29,7 @@ https://docs.djangoproject.com/en/1.8/ref/settings/ import os import dj_database_url import sys +from socket import gethostbyname BASE_DIR = os.path.dirname(os.path.abspath(__file__)) @@ -51,7 +52,7 @@ ALLOWED_HOSTS = [ # "lists.your-domain.org", # Add here all production URLs you may have. "mailman-web", - "172.19.199.3", + gethostbyname("mailman-web"), os.environ.get('SERVE_FROM_DOMAIN'), os.environ.get('DJANGO_ALLOWED_HOSTS'), ] @@ -60,7 +61,7 @@ ALLOWED_HOSTS = [ MAILMAN_REST_API_URL = os.environ.get('MAILMAN_REST_URL', 'http://mailman-core:8001') MAILMAN_REST_API_USER = os.environ.get('MAILMAN_REST_USER', 'restadmin') MAILMAN_REST_API_PASS = os.environ.get('MAILMAN_REST_PASSWORD', 'restpass') -MAILMAN_ARCHIVER_FROM = (os.environ.get('MAILMAN_HOST_IP', '172.19.199.2'),) +MAILMAN_ARCHIVER_FROM = (os.environ.get('MAILMAN_HOST_IP', gethostbyname(os.environ.get('MAILMAN_HOSTNAME', 'mailman-core'))),) # Application definition @@ -208,7 +209,7 @@ SERVER_EMAIL = 'root@{}'.format(hostname) # Change this when you have a real email backend EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend' -EMAIL_HOST = os.environ.get('SMTP_HOST', '172.19.199.1') +EMAIL_HOST = os.environ.get('SMTP_HOST', '') EMAIL_PORT = os.environ.get('SMTP_PORT', 25) EMAIL_HOST_USER = os.environ.get('SMTP_HOST_USER', '') EMAIL_HOST_PASSWORD = os.environ.get('SMTP_HOST_PASSWORD', '') diff --git a/tests/test.sh b/tests/test.sh index 6337b89..6babf74 100755 --- a/tests/test.sh +++ b/tests/test.sh @@ -27,10 +27,10 @@ docker logs mailman-core # Check to see if the core is working as expected. -docker exec mailman-core curl -u restadmin:restpass http://172.19.199.2:8001/3.1/system | grep "GNU Mailman" +docker exec mailman-core curl -u restadmin:restpass http://mailman-core:8001/3.1/system | grep "GNU Mailman" # Check to see if postorius is working. -docker exec mailman-web curl -L http://172.19.199.3:8000/postorius/lists | grep "Mailing List" +docker exec mailman-web curl -L http://mailman-web:8000/postorius/lists | grep "Mailing List" # Check to see if hyperkitty is working. -docker exec mailman-web curl -L http://172.19.199.3:8000/hyperkitty/ | grep "Available lists" +docker exec mailman-web curl -L http://mailman-web:8000/hyperkitty/ | grep "Available lists" diff --git a/web/README.md b/web/README.md index 01ede21..01d8caa 100644 --- a/web/README.md +++ b/web/README.md @@ -43,11 +43,12 @@ change them unless you know what you want. - `MAILMAN_REST_PASSWORD`: Mailman's REST API user's password. Default value is `restpass` -- `MAILMAN_HOST_IP`: IP of the Container from which Mailman will send emails to - hyperkitty (django). Set to `172.19.199.2` by default. +- `MAILMAN_HOSTNAME`: IP of the Container from which Mailman will send emails to + hyperkitty (django). Set to `mailman-core` by default. - `SMTP_HOST`: IP Address/hostname from which you will be sending - emails. Default value is `172.19.199.1`, which is the address of the Host OS. + emails. Default value is the container's gateway retrieved from: + /sbin/ip route | awk '/default/ { print $3 }' - `SMTP_PORT`: Port used for SMTP. Default is `25`. diff --git a/web/docker-entrypoint.sh b/web/docker-entrypoint.sh index 2657004..e2cd561 100755 --- a/web/docker-entrypoint.sh +++ b/web/docker-entrypoint.sh @@ -53,6 +53,11 @@ function check_or_create () { # END # } +# SMTP_HOST defaults to the gateway +if [[ ! -v SMTP_HOST ]]; then + export SMTP_HOST=$(/sbin/ip route | awk '/default/ { print $3 }') +fi + # Check if $SECRET_KEY is defined, if not, bail out. if [[ ! -v SECRET_KEY ]]; then echo "SECRET_KEY is not defined. Aborting." diff --git a/web/mailman-web/settings.py b/web/mailman-web/settings.py index f37f240..9356165 100644 --- a/web/mailman-web/settings.py +++ b/web/mailman-web/settings.py @@ -29,6 +29,7 @@ https://docs.djangoproject.com/en/1.8/ref/settings/ import os import dj_database_url import sys +from socket import gethostbyname BASE_DIR = os.path.dirname(os.path.abspath(__file__)) @@ -51,7 +52,7 @@ ALLOWED_HOSTS = [ # "lists.your-domain.org", # Add here all production URLs you may have. "mailman-web", - "172.19.199.3", + gethostbyname("mailman-web"), os.environ.get('SERVE_FROM_DOMAIN'), os.environ.get('DJANGO_ALLOWED_HOSTS'), ] @@ -61,7 +62,7 @@ MAILMAN_REST_API_URL = os.environ.get('MAILMAN_REST_URL', 'http://mailman-core:8 MAILMAN_REST_API_USER = os.environ.get('MAILMAN_REST_USER', 'restadmin') MAILMAN_REST_API_PASS = os.environ.get('MAILMAN_REST_PASSWORD', 'restpass') MAILMAN_ARCHIVER_KEY = os.environ.get('HYPERKITTY_API_KEY') -MAILMAN_ARCHIVER_FROM = (os.environ.get('MAILMAN_HOST_IP', '172.19.199.2'),) +MAILMAN_ARCHIVER_FROM = (os.environ.get('MAILMAN_HOST_IP', gethostbyname(os.environ.get('MAILMAN_HOSTNAME', 'mailman-core'))),) # Application definition @@ -227,7 +228,7 @@ SERVER_EMAIL = 'root@{}'.format(hostname) # Change this when you have a real email backend EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend' -EMAIL_HOST = os.environ.get('SMTP_HOST', '172.19.199.1') +EMAIL_HOST = os.environ.get('SMTP_HOST', '') EMAIL_PORT = os.environ.get('SMTP_PORT', 25) EMAIL_HOST_USER = os.environ.get('SMTP_HOST_USER', '') EMAIL_HOST_PASSWORD = os.environ.get('SMTP_HOST_PASSWORD', '')